3j5@cenet'., document v'iew 
MPEG DECODER PROVIDING MULTIPLE STANDARD OUTPUT SIGNALS 



Mm, 1/1 



Patent number: 
Publication date: 
Inventor: 

Applicant: 
Classification: 
- international: 



CN 1220800 
1999-06-23 

PHILLIPS LARRY; NAIMPALLY SAIPRASAD (JP); SITA 
RICHARD (JP) 

MATSUSHITA ELECTRIC IND CO LTD 
H04N5/44; H04N7/50; H04N7/26 



- european: 

Application number: CN 19980800295 19980311 
Priority number(s): US1 997004051 7P 19970312 



Also published as: 

~~ WO9841029 (A1) 
WO9841012 (A1) 
WO9841012 (A1) 
WO9841011 (A1) 
WO9841011 (A1) 

more » 



Abstract not available for CN 1220800 
Abstract of correspondent: WO984101 1 
A video decoder compliant with the Advanced 
Television Systems Standard (ATSC) includes 
circuitry which decodes an ATSC encoded image 
and employs a downcon vers ion process to 
produce a standard definition video signal. The 
video decoder includes a frequency-domain filter 
to reduce the resolution of the ATSC encoded 
signal. The video decoder downconversion 
system also includes a formatting section having 
vertical and horizontal filters, as well as 
resampling processing, to format the decoded 
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plurality of methods to convert the aspect ratio of 
the input signal for display on a display device 
having a different aspect ratio. In one mode, the 
system sequences through the possible methods 
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MPEG DECODER PROVIDING MULTIPLE STANDARD OUTPUT SIGNALS 

Description of correspondent: W09841 01 1 



HDTV DOWNCONVERSION SYSTEM 

This patent application claims the benefit of U.S. Provisional Application number 60/040,517 filed March 
12, 1997. 

The entire disclosure of U.S. Provisional Application No. 60/040,517 is expressly incorporated herein by 
reference. 

FIELD OF THE INVENTION 

This invention relates to a decoder for receiving, decoding and conversion of frequency domain encoded 
signals, e.g. MPEG-2 encoded video signals, into standard output video signals, and more specifically to a 
decoder which converts and formats an encoded high resolution video signal to a decoded lower 
resolution output video signal. 

BACKGROUND OF THE INVENTION 

In the United States a standard, the Advanced Television System Committee (ATSC) standard defines 
digital encoding of high definition television (HDTV) signals. 

A portion of this standard is essentially the same as the MPEG-2 standard, proposed by the Moving 
Picture Experts Group (MPEG) of the International Organization for 

Standardization (ISO). The standard is described in an International Standard (IS) publication entitled, 
"Information Technology - Generic Coding of Moving Pictures and 

Associated Audio, Recommendation H.626", ISO/IEC 13818-2, IS, 1 1/94 which is available from the ISO 
and which is hereby incorporated by reference for its teaching on the MPEG-2 digital video coding 
standard. 

The MPEG-2 standard is actually several different standards. In MPEG-2 several different profiles are 
defined, each corresponding to a different level of complexity of the encoded image. For each profile, 
different levels are defined, each level corresponding to a different image resolution. One of the MPEG-2 
standards, known as Main Profile, 

Main Level is intended for coding video signals conforming to existing television standards (i.e., NTSC and 
PAL). Another standard, known as Main Profile, High Level is intended for coding high-definition television 
images. Images encoded according to the 

Main Profile, High Level standard may have as many as 1 ,1 52 active lines per image frame and 1 ,920 
pixels per line. 

The Main Profile, Main Level standard, on the other hand, defines a maximum picture size of 720 pixels 
per line and 567 lines per frame. At a frame rate of 30 frames per second, signals encoded according to 
this standard have a data rate of 720 * 567 * 30 or 12,247,200 pixels per second. By contrast, images 
encoded according to the Main 

Profile, High Level standard have a maximum data rate of 1,152 * 1,920 * 30 or 66,355,200 pixels per 
second. This data rate is more than five times the data rate of image data encoded according to the Main 
Profile Main Level standard. The standard for 

HDTV encoding in the United States is a subset of this standard, having as many as 1 ,080 lines per frame, 
1 ,920 pixels per line and a maximum frame rate, for this frame size, of 30 frames per second. The 
maximum data rate for this standard is still far greater than the maximum data rate for the Main Profile, 
Main Level standard. 

The MPEG-2 standard defines a complex syntax which contains a mixture of data and control information. 
Some of this control information is used to enable signals having several different formats to be covered 
by the standard. These formats define images having differing numbers of picture elements (pixels) per 
line, differing numbers of lines per frame or field and differing numbers of frames or fields per second. In 
addition, the basic syntax of the MPEG-2 Main Profile defines the compressed MPEG-2 bit stream 
representing a sequence of images in five layers, the sequence layer, the group of pictures layer, the 
picture layer, the slice layer, and the macroblock layer. Each of these layers is introduced with control 
information. Finally, other control information, also known as side information, (e.g. frame type, 
macroblock pattern, image motion vectors, coefficient zig-zag patterns and dequantization information) are 
interspersed throughout the coded bit stream. 

Format conversion of encoded high resolution Main Profile, High Level pictures to lower resolution Main 
Profile, High Level pictures; Main Profile, Main Level pictures, or other lower resolution picture formats, 
has gained increased importance for a) providing a single decoder for use with multiple existing video 
formats, b) providing an interface between Main Profile, high level signals and personal computer monitors 



http://v3. espacenet. com/textdes?DB=EP0D0C&IDX=CN1220800&QPN=CN1220800 



2004-3-18 



sp@cenet description view 
or existing consumer television receivers, and c) reducing implementation costs of HDTV. 



TO, 2/26 



For example, conversion allows replacement of expensive high definition monitors used with Main Profile, 
J High Level encoded pictures with inexpensive existing monitors which have a lower picture resolution to 
support, for example, Main Profile, Main Level encoded pictures, such as NTSC or 525 progressive 
monitors. One aspect, down conversion, converts a high definition input picture into lower resolution 
picture for display on the lower resolution monitor. 

To effectively receive the digital images, a decoder should process the video signal information rapidly. To 
be optimally effective, the decoding systems should be relatively inexpensive and yet have sufficient 
power to decode these digital signals in real time. 

Consequently, a decoder which supports conversion into multiple low resolution formats must minimize 
processor memory. 

SUMMARY OF THE INVENTION 

The present invention is embodied in a digital video signal processing system which receives, decodes 
and displays video signals that have been encoded in a plurality of different formats. The system includes 
a digital video decoder which may be controlled to decode the encoded video signal and, optionally, 
provide a reduced resolution version of the decoded video signal. The system processes the received 
encoded video signal to determine the format and resolution of the image which would be produced if the 
signal were decoded. The system includes a controller which receives the determined format and 
resolution information and which also receives information concerning the format and resolution of a 
display device on which the received image will be displayed. The controller then generates signals to 
cause the digital video decoder to provide an analog video signal having a resolution and aspect ratio that 
is appropriate for the display device. 

According to one aspect of the invention, the encoded video signals are encoded using a frequency- 
domain transform operation and the digital video decoder includes a low-pass filter which operates on the 
frequency-domain transformed digital video signal. 

According to another aspect of the invention, digital video decoder is coupled to a programmable spatial 
filter which is responsive to a control signal provided by the controller to resample the decoded digital 
video signal provided by the digital video decoder to produce a digital video signal which conforms to the 
aspect ratio and resolution of the display device. 

According to another aspect of the invention, the digital video signal is encoded according to using an 
encoding technique specified by the moving pictures experts group (MPEG) and the aspect ratio and 
resolution of the encoded video signal are extracted from the header of a packetized elementary stream 
(PES) packet received by the digital video decoder. 

According to another aspect of the invention, the digital video signal is encoded according to using an 
encoding technique specified by the moving pictures experts group (MPEG) and the aspect ratio and 
resolution of the encoded video signal are extracted from a sequence header of a video bit-stream 
received by the digital video decoder. 

According to another aspect of the invention, the system includes a user input device through which a user 
may configure the system to produce an output video signal which is compatible with the display device. 

According to another aspect of the invention, the system includes apparatus which automatically 
determines the aspect ratio and resolution of the display device. 

According to another aspect of the invention, the system includes apparatus which sequentially produces 
video signals corresponding to a plurality of display device types and is responsive to a selection signal 
provided by a user to identify one of the display types as corresponding in resolution and aspect ratio to 
the display device. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other features and advantages of the present invention will become apparent from the following 
detailed description, taken in conjunction with the accompanying drawings, wherein: 
Figure 1 A is a high level block diagram of a video decoding and format conversion system according to an 
exemplary embodiment of the present invention. 

Figure 1 B is a high level block diagram showing the functional blocks of the ATV 

Video Decoder including an interface to external Memory as employed in an exemplary embodiment of the 
present invention. 

Figure 2A is a high level block diagram of a video decoder of the prior art. 
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Figure 2B is a high level block diagram of the down conversion system as employed by an exemplary 
embodiment of the present invention. 

Figure 2C is a block diagram which illustrates a configuration of the decoder shown in Figure 2B which is 
used to decode a video signal in1 1251 format including a downconversion by factor of 3 to 525P/525I 
format. 

Figure 2D is a block diagram which illustrates a configuration of the decoder shown in Figure 2B which is 
used to decode a video signal in 750P format including a downconversion by factor of 2 to 525P/525I 
format. 

Figure 3A is a pixel chart which illustrates subpixel positions and corresponding predicted pixels for the 3:1 
and 2:1 exemplary embodiments of the present invention. 

Figure 3B is a flow-chart diagram which shows the upsampling process which is performed for each row of 
an input macroblock for an exemplary embodiment of the present invention. 

Figure 4 is a pixel chart which illustrates the multiplication pairs for the first and second output pixel values 
of an exemplary embodiment of a block mirror filter. 

Figure 5 is a block diagram which illustrates an exemplary implementation of the filter for down-conversion 
for a two-dimensional system processing the horizontal and vertical components implemented as 
cascaded one-dimensional IDCTs. 

Figure 6A is a macroblock diagram which shows the input and decimated output pixels for 4:2:0 video 
signal using 3:1 decimation. 

Figure 6B is a pixel block diagram which shows the input and decimated output pixels for 4:2:0 video 
signal using 2:1 decimation. 

Figure 6C is a macroblock diagram which illustrates a merging process of two macroblocks into a single 
macroblock for storage in memory for downconversion by 2 horizontally. 

Figure 6D is a macroblock diagram which illustrates a merging process of three macroblocks into a single 
macroblock for storage in memory for downconversion by 3 horizontally. 

Figure 7A is a block diagram illustrating a vertical programmable filter of one embodiment of the present 
invention. 

Figure 7B is a pixel diagram which illustrates the spatial relationships between vertical filter coefficients 
and a pixel sample space of lines of the vertical programmable filter of Figure 7A. 

Figure 8A is a block diagram illustrating a horizontal programmable filter of one embodiment of the present 
invention. 

Figure 8B is a pixel diagram which illustrates spatial relationships between horizontal filter coefficients and 
pixel sample values of one embodiment of the present invention. 

Figure 9A is a graph of pixel number versus resampling ratio which illustrates a resampling ratio profile of 
an exemplary embodiment of the present invention. 

Figure 9B is a graph which shows a first ratio profile for mapping a 4:3 picture onto a 16:9 display. 

Figure 9C is a graph which shows a second ratio profile for mapping a 4:3 picture onto a 16:9 display. 

Figure 9D is a graph which shows a first ratio profile for mapping a 16:9 picture onto a 4:3 display. 

Figure 9E is a graph which shows a second ratio profile for mapping a 16:9 picture onto a 4:3 display. 

Figure 10 is a chart of image diagrams which illustrates the effect of using resampling ratio profiles 
according to an exemplary embodiment of the present invention. 

Figurel 1 A is a high level block diagram illustrating the display section of the 
ATV Video Decoder of an exemplary embodiment of the present invention. 

Figurel 1 B is a block diagram which illustrates a 27 MHz Dual output mode of an exemplary embodiment 
of the present invention which, for which the video data is 525P or 525I, a first processing chain provides 
video data to a 27 MHz DAC well as to an 
NTSC Encoder. 
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FigurellC is a block diagram which illustrates that, in the 27 MHz single output mode of an exemplary 
ejnbodiment of the present invention, only a525l video signal is provided to a NTSC encoder. 

Figurel 1 D is a block diagram which illustrates a 74 MHz /27 MHz mode of an exemplary embodiment of 
the present invention in which the output format matches the input format and the video data is provided to 
either a 27 MHz DAC or 74 MHz DAC depending on the input format. 

Figure 12 is a high level block diagram of the video decoder having high bandwidth memory as employed 
by an exemplary embodiment of the present invention to decode ATSC video signals. 

DETAILED DESCRIPTION 
System Overview 

The exemplary embodiments of the invention decode conventional HDTV signals which have been 
encoded according to the MPEG-2 standard and in particular, the Main 

Profile High Level(MP&commat;HL) and the Main Profile Main Level(MP&commat;ML) MPEG-2 
standards, and provides the decoded signals as video signals having a lower resolution than that of the 
received HDTV signals and having a selected one of multiple formats. 

The MPEG-2 Main Profile standard defines a sequence of images in five levels: the sequence level, the 
group of pictures level, the picture level, the slice level, and the macroblock level. Each of these levels 
may be considered to be a record in a data stream, with the later-listed levels occurring as nested sub- 
levels in the earlier listed levels. The records for each level include a header section which contains data 
that is used in decoding its sub-records. 

Each macroblock of the encoded HDTV signal contains six blocks and each block contains data 
representing 64 respective coefficient values of a discrete cosine transform (DCT) representation of 64 
picture elements (pixels) in the HDTV image. 

In the encoding process, the pixel data may be subject to motion compensated differential coding prior to 
the discrete cosine transformation and the blocks of transformed coefficients are further encoded by 
applying run-length and variable length encoding techniques. A decoder which recovers the image 
sequence from the data stream reverses the encoding process. This decoder employs an entropy decoder 
(e.g. a variable length decoder), an inverse discrete cosine transform processor, a motion compensation 
processor, and an interpolation filter. 

The video decoder of the present invention is designed to support a number of different picture formats, 
while requiring a minimum of decoding memory for downconversion of high resolution encoded picture 
formats, for example, 48 Mb of 

Concurrent Rambus dynamic random access memory (Concurrent RDRAM). 

Figure 1A shows a system employing an exemplary embodiment of the present invention for receiving and 
decoding encoded video information atMP&commat;HL or atMP&commat;ML, formatting the decoded 
information to a user selected output video format (which includes both video and audio information), and 
interfaces for providing the formatted video output signals to display devices. The exemplary embodiments 
of the present invention are designed to support all ATSC video formats; and in a Down 
Conversion (DC) mode the present invention receives any MPEG Main Profile video bitstream 
(constrained by FCC standards) and provides a 525P, 525I or NTSC format picture. 

The exemplary system of Figure 1 A includes a front end interface 100, a video decoder section 120 and 
associated Decoder Memory 130, a primary video output interface 140, an audio decoder section 160, an 
optional computer interface 110, and an optional NTSC video processing section 150. 

Referring to Figure 1 A, the exemplary system includes a front end interface 100, having a transport 
decoder and processor 102 with associated memory 103. Also included may be an optional multiplexer 
101 for selecting received control information and computer generated images from the computer interface 
1 10 at, for example, the IEEE 1394 link layer protocol or for recovering an encoded transport stream from 
a digital television tuner (not shown). The transport decoder 102 converts the received compressed data 
bit stream from the communication channel bit stream into compressed video data, which may be, for 
example, packetized elementary streams (PES) packets according to MPEG-2 standard. The transport 
decoder may provide either the PES packets directly, or may further convert the PES packets mt o one or 
more elen^t^^ffea^rins : 

The video decoder section includes an ATV Video Decoder 121 and phase-locked loop (PLL) 122. The 
ATV video Decoder 121 receives an elementary stream or video (PES) packets from the front end 
interface 100, from the front end interface and converts the packets to the elementary stream. A front end 
picture processor of the ATV Video 

Decoder 121 then decodes the elementary streams according to the encoding method used, to provide 
luminance and chrominance pixel information for each image picture. The 
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PLL 122 synchronizes the audio and video processing performed by the system shown in 
Figure 1A. 

The ATV Video Decoder 121 further includes a memory subsystem to control decoding operations using 
an external memory which provides image picture information and a display section to process decoded 
picture information into a desired picture format. 

The ATV Video Decoder 121 employs the Decoder Memory 130 to process the encoded video signal. The 
Decoder Memory 130 includes memory units 131, 132, 133, 134, 135 and 136, which may each be a 16 
Mb RDRAM memory. Exemplary embodiments the present invention are subsequently described with 
respect to, and implemented within, the video decoder section 120 and Decoder Memory 130. 

The primary video output interface 140 includes a first Digital to Analog (D/A) converter (DAC) 141 (which 
actually has three D/A units for the luminance signal and the 

CR and CB chrominance signals) which may operate at 74 MHz, followed by a filter 142. 

This interface produces analog video signals having a1 1251 or 750P format. The interface 140 and also 
includes a second (D/A) converter (DAC) 143 (also with three D/A units for luminance signal and CR and 
CB chrominance signals) which may operate at 27 

MHz, followed by a filter 142 for video signals having a525l or 525P format. The primary video output 
interface 140 converts the digitally encoded video signals having a desired format, creates an analog 
video signal having chrominance and luminance components with the desired format using a (D/A) 
converter, and filters the analog video signal to remove sampling artifacts of the D/A conversion process. 

The audio decoder section 160 includes an AC3 Audio decoder 162 which provides audio signals at 
output ports 163 and 164, and optional 6-2 channel down mixing processor 161 to provide 2 channel audio 
signals at output port 165. The audio processing ofMP&commat;HL MPEG-2 standard audio signal 
components from encoded digital information to analog output at output ports 163, 164 and 165 is well 
known in the art, and an audio decoder suitable for use as the decoder 160 is a ZR38500 Six Channel 
Dolby 

Digital Surround Processor, available from the Zoran Corporation of Santa Clara, CA. 

The optional computer interface 110 transmits and receives computer image signals which conform, for 
example, to the IEEE 1 394 standard. The computer interface 110 includes a physical layer processor 111 
and link layer processor 112. The physical layer processor 1 1 1 converts electrical signals from output port 
113 into received computer generated image information and control signals, and provides these signals, 
for decoding by the link layer processor 112 into IEEE 1394 formatted data. The physical layer processor 
1 1 1 also converts received control signals encoded by the link layer processor 112 originating from the 
transport decoder 102 into electrical output signals according to the IEEE 1394 standard. 

The NTSC video processing section 150 includes an optional ATV-NTSC 

Downconverston processor 151 which converts the analog HDTV signal provided by the filter 142 into a 
525 I signal. This conversion between standards is known in the art and may be accomplished using 
spatial filtering techniques such as those disclosed in, for example, U.S. Patent 5,613,084 to Hau et al. 
entitled INTERPOLATION FILTER 

SELECTION CIRCUIT FOR SAMPLE RATE CONVERSION USING PHASE 

QUANTIZATION, which is incorporated herein by reference. In the exemplary embodiment of the 

invention, this processing section is used only when the decoder processes a1080l or1 1251 signal. 

The NTSC encoder 1 52 receives a525l analog signal either from the processor 1 51 or directly from the 
decoder 120, and converts the signal to the NTSC formatted video signal at output ports 153 (S-video) 
and 154 (composite video). 

Video Decoder Section Employing Decoder Memory 

Figure 1B is a high level block diagram showing the functional blocks of the ATV 

Video Decoder 121 including an interface to external Memory 130 as employed in an exemplary 

embodiment of the present invention. The ATV Video Decoder 121 includes a 

Picture Processor 171, a Macroblock Decoder 172, a Display section 173, and a Memory subsystem 174. 
The Picture processor 171 receives, stores and partially decodes the incoming MPEG-2 video bitstream 
and provides the encoded bitstream, on-screen display data, and motion vectors, which may be stored in 
memory 130 under the control of the 

Memory subsystem 174. The Macroblock Decoder 172 receives the encoded bitstream, motion vectors, 
and stored motion compensation reference image data, if predictive encoding is used, and provides 
decoded macroblocks of the encoded video image to the memory subsystem 174. The Display Section 
173 retrieves the decoded macroblocks from the Memory subsystem 174 and formats these into the video 
image picture for display. The operation of these sections is described in detail below. 

a) Main Profile Format Support for Picture Processing 

The ATV video decoder 121 of the present invention is designed to support all 
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ATSC video formats. For simplicity, the operation of the ATV video decoder 121 is termed Down 
Conversion (DC), and ATV video decoder 121 receives any MPEG Main 

Profile video bitstreams shown in Table 1 and provides a 525P, 525I or NTSC format video signal. For the 
exemplary video decoder of Figure 1 A, in DC mode, any HDTV or 

SDTV signal is decoded and a display output signal provided at either of two ports, with port one providing 
either a progressive or interlaced image, and port two providing an interlaced image. 

Table 1 

Video Bitstream Formats 
EMI12.1 



<tb> <SEP> Number <SEP> and <SEP> Format <SEP> Horizontal <SEP> Vertical <SEP> Aspect <SEP> 
Ratio <SEP> Frame <SEP> rate <SEP> (Hz) 

<tb>(1)1 1251 <SEP> 1920 <SEP> 1080 <SEP> 16x9 <SEP> 30, <SEP> 29.97 

<tb> (2) <SEP> 1125P <SEP> 1920 <SEP> 1080 <SEP> 16x9 <SEP> 30, <SEP> 29.97, <SEP> 24, 
<SEP> 23.98 _ 
<tb> (3) <SEP> 750P <SEP> 1280 <SEP> 720 <SEP> 16x9 <SEP> 60, <SEP> 59.94, <SEP> 30, <SEP> 

29.97, 

<tb> <SEP> 24, <SEP> 23.98 

<tb> (4) <SEP> 525P <SEP> 704 <SEP> 480 <SEP> 16x9 <SEP> 60, <SEP> 59.94, <SEP> 30, <SEP> 
29.97, 

<tb> <SEP> 24, <SEP> 23.98 

<tb> (5) <SEP> 525P <SEP> 704 <SEP> 480 <SEP> 4x3 <SEP> 60, <SEP> 59.94, <SEP> 30, <SEP> 
29.97, 

<tb> <SEP> 24, <SEP> 23.98 <SEP> 

<tb> (6) <SEP> 525P <SEP> 640 <SEP> 480 <SEP> 4x3 <SEP> 60, <SEP> 59.94, <SEP> 30, <SEP> 
29.97, 

<tb> <SEP> 24, <SEP> 23.98 <SEP> 

<tb> (7) <SEP> 5251 <SEP> 704 <SEP> 480 <SEP> 16x9 <SEP> 30, <SEP> 29.97 <SEP> 
<tb> (8) <SEP> 5251 <SEP> 704 <SEP> 480 <SEP> 4x3 <SEP> 30, <SEP> 29.97 <SEP> 
<tb> (9) <SEP> 5251 <SEP> 640 <SEP> 480 <SEP> 4x3 <SEP> 30, <SEP> 29.97 
<tb> 

In DC mode, low pass filtering of the high frequency components of the Main 

Level picture occurs as part of the decoding process to adjust the resolution of the high resolution picture 
to a format having a lower resolution. This operation includes both horizontal and vertical filtering of the 
high resolution picture. Note that in DC Mode, the display format conversion may display 16x9 aspect ratio 
sources on 4x3 displays, and vice-versa. This process is described subsequently with reference to the 
display section of the video decoder section 120. Table 2 gives the supported primary and secondary 
output picture formats for the respective input bitstreams of Table 1 : 
Table 2 

DC Supported Video Formats 
EMI13.1 



<tb> Number <SEP> and <SEP> Format <SEP> Primary <SEP> Output <SEP> Secondary <SEP> 

Output <SEP> Display <SEP> Clock <SEP> (MHz) 

<tb> <SEP> Format <SEP> Format 

<tb> (1)1 1251 <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> (2) <SEP> 1 125P <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> (3) <SEP> 750P <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> (4) <SEP> 525P <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> (5) <SEP> 525P <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> (6) <SEP> 525P <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> (7) <SEP> 5251 <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> (8) <SEP> 5251 <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> (9) <SEP> 5251 <SEP> 525P <SEP> 5251 <SEP> 27.00 

<tb> b) Decoding. Downconversion andDownsampling 1) Overview 

Figure 2A is a high level block diagram of a typical video decoding system of the prior art which processes 
an MPEG-2 encoded picture. The general methods used to decode an MPEG-2 encoded picture, without 
subsequent processing, downconversion or format conversion, are specified by the MPEG-2 standard. 
The video decoding system includes an entropy decoder (ED) 21 1 , which may include a parser 209, a 
variable length decoder (VLD) 210 and run length decoder 212. The system also includes an inverse 
quantizer 214, and inverse discrete cosine transform (IDCT) processor 218. A Controller 207 controls the 
various components of the decoding system responsive to the control information retrieved from the input 
bit stream by the ED 21 1. For processing of prediction images, the system further includes a memory 199 
having reference frame memory 222, summing network 230, and Motion Compensation Processor 206a 
which may have a motion vector processor 221 and half-pixel generator 228. 
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The controller 207 is coupled to an infrared receiver 208 which receives command signals provided by, for 
example, a user remote control device. The controller 207 decodes these commands and causes the 
remainder of the system shown in Figure 2A to perform the specified command. In the exemplary 
embodiment of the invention, the system shown in Figure 2A includes a set-up mode in which the user 
may specify a configuration for the system. In one exemplary embodiment of the invention, this 
configuration may include the specification of a display device type. It is contemplated that the display 
device type may be specified in terms of display resolution and aspect ratio. The user may specify the 
display type by selecting a particular display aspect ratio and resolution from a menu of possible choices 
or by causing the system to enter a mode in which signals corresponding to different display formats are 
successively provided to the display device and the user is asked to indicate, via the remote control 
device, which display is most pleasing. During normal operation, the controller 207 also receives 
information on the resolution and aspect ratio of the encoded video signal from the parser 209 of the ED 
211 . Using this information and the stored information relating to the resolution and aspect ratio of the 
display device, the controller 207 automatically configures the system to process the received encoded 
signal to produce an analog output signal appropriate for display on the display device. 

The parser 209 scans the received bit stream for MPEG start codes. These codes include a prefix which 
has a format of 23 consecutive zero-valued bits followed by a single bit having a value of one. The start 
code value follows this prefix and identifies the type of record that is being received. In the exemplary 
embodiment of the invention, when the parser 209 stores the bit stream into the memory 199 and the bit 
stream is then supplied from the memory 199 to the VLD for further processing. In the block diagram 
shown in Figure 2A, this has been shortened to show the parser providing the bit stream directly to the 
VLD. 

When the parser 209 finds a start code in the bit stream, it passes the bit stream onto the memory 
headers. When the controller 207 finds a sequence start code, it accesses information in the sequence 
header which indicates the aspect ratio and resolution of the image sequence that is represented by the 
encoded sequence. According to the MPEG standard, this information immediately follows the sequence 
start code in the sequence header. 

Information on the display format of the encoded video signal (i.e. its resolution and aspect ratio) is also 
contained in the headers of the packetized elementary stream (PES) packets. It is contemplated that in 
another exemplary embodiment of the invention, the parser 209 may receive PES packets, strip the 
headers from them to reconstruct the bit stream and pass this header information, including the display 
format of the received video signal, to the controller 207. 

As described below, the controller 207 uses information on the display format of the received video signal 
and information on the display format of the display device (not shown) which is connected to the decoder 
system to automatically or semiautomatically adjust the processing of the received video signal for proper 
display on the display device. 

The VLD 210 receives the encoded bit stream from the parser 209 via the VBV buffer (not shown) in the 
memory 199, and reverses the encoding process to produce macroblocks of quantized frequency-domain 
(DCT) coefficient values. The VLD 210 also provides control information including motion vectors 
describing the relative displacement of a matching marcoblock in a previously decoded image which 
corresponds to a macroblock of the predicted picture that is currently being decoded. The Inverse 
Quantizer 214 receives the quantized DCT transform coefficients and reconstructs the quantized DCT 
coefficients for a particular macroblock. The quantization matrix to be used for a particular block is 
received from the ED 21 1 . 

The IDCT processor 218 transforms the reconstructed DCT coefficients to pixel values in the spatial 
domain (for each block of 8 X 8 matrix values representing luminance or chrominance components of the 
macroblock, and for each block of 8 X 8 matrix values representing the differential luminance or differential 
chrominance components of the predicted macroblock). 

If the current macroblock is not predictively encoded, then the output matrix values provided by the IDCT 
Processor 21 8 are the pixel values of the corresponding macroblock of the current video image. If the 
macroblock is interframe encoded, the corresponding macroblock of the previous video picture frame is 
stored in memory 199 for use by the Motion Compensation processor 206. The Motion Compensation 
Processor 206 receives a previously decoded macroblock from memory 199 responsive to the motion 
vector, and then adds the previous macroblock to the current IDCT macroblock (corresponding to a 
residual component of the present predictively encoded frame) in summing network 230 to produce the 
corresponding macroblock of pixels for the current video image, which is then stored into the reference 
frame memory 222. 

Figure 2B is a high level block diagram of the down conversion system of one exemplary embodiment of 
the present invention employing such a DCT filtering operation, and which may be employed by an 
exemplary embodiment of the present invention in DC mode. As shown in Figure 2B, the down conversion 
system includes a variable length decoder (VLD) 210, a run-length (R/L) decoder 212, an inverse 
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quantizer 214, and inverse discrete cosine transform (IDCT) processor 218. In addition, the down 
conversion system includes a Down Conversion filter 216 for filtering encoded pictures and a Down 
Sampling processor 232. While the following describes the exemplary embodiment for aMP&commat;HL 
encoded input, the present invention may be practiced with any similarly encoded high-resolution image 
bit stream. 

The down conversion system also includes a Motion Compensation Processor 206b including a Motion 
Vector (MV) Translator 220, a Motion Block Generator 224 including an Up-Sampling Processor 226, Half- 
Pixel Generator 228, and a Reference Frame 
Memory 222. 

The system of the first exemplary embodiment of Figure 2B also includes a 

Display Conversion Block 280 having a Vertical Programmable Filter (VPF) 282 and 

Horizontal Programmable Filter (HZPF) 284. The Display Conversion Block 280 converts downsampled 

images into images for display on a particular display device having a lower resolution than the original 

image, and is described in detail subsequently in section d)(2) on Display Conversion. 

The Down Conversion Filter 216 performs a lowpass filtering of the high resolution (e.g. Main Profile, High 
Level DCT) coefficients in the frequency domain. 

The Down Sampling Process 232 eliminates spatial pixels by decimation of the filtered 
Main Profile, High Level picture to produce a set of pixel values which can be displayed on a monitor 
having lower resolution than that required to display aMP&commat;HL picture. The exemplary Reference 
Frame Memory 222 stores the spatial pixel values corresponding to at least one previously decoded 
reference frame having a resolution corresponding to the down-sampled picture. For interframe encoding, 
the MV Translator 220 scales the motion vectors for each block of the received picture consistent with the 
reduction in resolution, and the High Resolution Motion Block Generator 224 receives the low resolution 
motion blocks provided by the Reference Frame Memory 222, upsamples these motion blocks and 
performs half-pixel interpolation as needed to provide motion blocks which have pixel positions that 
correspond to the decoded and filtered differential pixel blocks. 

Note that in the down conversion system of Figure 1B the downsampled images are stored rather than 
high definition images, resulting in a considerable reduction of memory required for storing reference 
images. 

The operation of an exemplary embodiment of the down-conversion system of the present invention for 
intra-frame encoding is now described. TheMP&commat;HL bit-stream is received and decoded by VLD 
210. In addition to header information used by the HDTV system, the VLD 210 provides DCT coefficients 
for each block and macroblock, and motion vector information. The DCT coefficients are run length 
decoded in the R/L decoder 212 and inverse quantized by the inverse quantizer 214. 

Since the received video image represented by the DCT coefficients is a high resolution picture, the 
exemplary embodiment of the present invention employs lowpass filtering of the DCT coefficients of each 
block before decimation of the high resolution video image. The inverse quantizer 214 provides the DCT 
coefficients to the DCT filter 216 which performs a lowpass filtering in the frequency domain by weighting 
the DCT coefficients with predetermined filter coefficient values before providing them to the 
IDCT processor 218. For one exemplary embodiment of the present invention, this filter operation is 
performed on a block by block basis. 

The IDCT processor 218 provides spatial pixel sample values by performing an inverse discrete cosine 
transform of the filtered DCT coefficients. The Down Sampling processor 232 reduces the picture sample 
size by eliminating spatial pixel sample values according to a predetermined decimation ratio; therefore, 
storing the lower resolution picture uses a smaller frame memory compared to that which would be 
needed to store the higher resoiutionMP&commat;HL picture. 

The operation of an exemplary embodiment of the down-conversion system of the present invention for 
predicted frames of the encoding standard is now described. In this example, the current received image 
DCT coefficients represent the DCT coefficients of the residual components of the predicted image 
macroblocks, which is now referred to as a predicted frame (P-frame) for convenience. In the described 
exemplary embodiment, the horizontal components of the motion vectors for a predicted frame are scaled 
since the low resolution reference pictures of previous frames stored in memory do not have the same 
number of pixels as the high resolution predicted frame(MP&commat;HL). 

Referring to Figure 2B, the motion vectors of theMP&commat;HL bit stream provided by the VLD 210 are 
provided to the MV Translator 220 Each motion vector is scaled by the 

MV Translator 220 to reference the appropriate prediction block of the reference frame of a previous 
image stored in reference frame memory 222 of memory 199. The size (number of pixel values) in the 
retrieved block is smaller than block provided by the 

IDCT processor 218; consequently, the retrieved block is upsampled to form a prediction block having the 
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same number of pixels as the residual block provided by the IDCT 
Processor 218 before the blocks are combined by the summing network 230. 

The prediction block is upsampled by the Up-Sampling Processor 226 responsive to a control signal from 
the MV Translator 220 to generate a block corresponding to the original high resolution block of pixels, 
and then half pixel values are generated - if indicated by the motion vector for the up-sampled prediction 
block in the Half Pixel 

Generator 228 - to ensure proper spatial alignment of the prediction block. The upsampled and aligned 
prediction block is added in summing network 230 to the current filtered block, which is, for this example, 
the reduced resolution residual component from the prediction block. All processing is done on a 
macroblock by macroblock basis. After the motion compensation process is complete for the current high- 
resolution macroblock, the reconstructed macroblock is decimated accordingly by the Down Sampling 
Processor 232. This process does not reduce the resolution of the image but simply removes redundant 
pixels from the low resolution filtered image. 

Once the downsampled macroblocks for an image are available, the Display 

Conversion Block 280 adjusts the image for display on a low resolution television display unit by filtering 
the vertical and horizontal components of the downsampled image in VPF 282 and HZPF 284 
respectively. 

The relationship between the functional blocks of the ATV Video Decoder 121 of 

Figure 1A and Figure 1B is now described. The picture processor 171 of Figure 1B receives the video 

picture information bitstreams. The Macroblock Decoder 172 includes 

VLD 210, inverse quantizer 214, the DCT filter 216, IDCT 21 8, summing network 230, and the motion 
compensated predictors 206a and 206b. The picture processor 171 may share the VLD 210. External 
Memory 130 corresponds to memory 199, with 16Mb 
RDRAM 131-136 containing the reference frame memory 222. 

Figure 2C illustrates the operation of the system in DC mode, converting an1125l signal to 525P/525I 
format. In this scenario, after low pass filtering by DCT filter 216 as described above with reference to 
Figure 2B, the system down samples the high resolution signal by a factor of 3, and stores the pictures in 
the 48 Mb memory as 640H and 1 080 

V, interlaced. For this system, the motion compensation process upsamples the stored pictures by a factor 
of 3 (as well as translation of the received motion vectors) before motion-predictive decoding is 
accomplished. Also, the picture is filtered horizontally and vertically for display conversion. 

Figure 2D similarly illustrates the relationship between DC mode format downconversion from 750P to 
525P/525I format. This conversion operates in the same way as the1125l to 525P/525I conversion except 
that downsampling for memory storage, and upsampling for motion compensation, is by a factor of 2. 

2) Macroblock Prediction for Downconversion 

For the exemplary downconversion process, since the reference frames of the previous images are down 
sized in the horizontal direction, the received motion vectors pointing to these frames may also be 
translated according to the conversion ratio. The following describes the motion translation for the 
luminance block in the horizontal direction. One skilled in the art could easily extend the following 
discussion to motion translation in the vertical direction if desired. Denoting x and y as the current 
macroblock address in the original image frame, Dx as the horizontal decimation factor andmvX as the 
half pixel horizontal motion vector of the original image frame, the address of the top left pixel of the 
motion block in the original image frame, denoted as XH in the half pixel unit, is given by (1): 
XH=2x+mv, (1) 

The pixel corresponding to the motion block starts in the down-sampled image, and has an address 
denoted as x* and y* may be determined using equation (2). 

XH * = = , y = y (2) 

The division of equation (2) is an integer division with truncation. 

Because the exemplary filter 216 and Down Sampling Processor 232 only reduce the horizontal 
components of the image, the vertical component of the motion vector is not affected. For the 
chrominance data, the motion vector is one-half of a luminance motion vector in the original picture. 
Therefore, definitions for translating the chrominance motion vector may also use the two equations (1 ) 
and (2). 

Motion prediction is done by a two step process: first, pixel accuracy motion estimation in the original 
image frame may be accomplished by upsampling of downsampled image frame in the Up Sampling 
Processor 226 of Figures 2A and 2B, then the half pixel Generator 228 performs a half pixel interpolation 
by averaging of nearest pixel values. 

The reference image data is added to output data provided by the IDCT processor 218. Since the output 
values of the summing network 230 correspond to an image having a number of pixels consistent with a 

ittp://v3. espacenet. com/textdes?DB=EP0D0C&IDX=CN1220800&QPN=CN1220800 



;p@cenet description view 10/26 

high resolution format, these values may be downsampled for display on a display having a lower 
resolution. Downsampling in the 

Down Sampling processor 232 is substantially equivalent to subsampling of an image frame, but 
adjustments may be made based upon the conversion ratio. For example, in the case of 3:1 
downsampling, the number of horizontally downsampled pixels are 6 or 5 for each input macroblock, and 
the first downsampled pixels are not always first pixel in the input macroblock. 

After acquiring the correct motion prediction block from the down-sampled image, upsampling is used to 
get the corresponding prediction block in the high resolution picture. Consequently, subpixel accuracy in 
motion block prediction is desirable in the down sampled picture. For example, using 3:1 decimation, it is 
desirable to have 1/3 (or 1/6) subpixel accuracy in the down-converted picture for proper motion 
prediction. The subpixel which is a first pixel required by the motion vector, in addition to the downsampled 
motion block, is determined. Then, subsequent subpixel positions are determined using modulo arithmetic 
as described in the following. The subpixel positions are denoted asx5 as given in equation (3): 
EMI21.1 

where"%" represents modulo division. 

For example, the ranges ofx5 are 0, 1, 2 for 3:1 upsampling and 0, 1 for 2:1 upsampling. Figure 3A shows 
subpixel positions and corresponding 17 predicted pixels for the 3:1 and 2:1 examples, and Table 3 gives 
the legend for Figure 3A. 

Table 3 
EMI21.2 



<tb> Symbol <SEP> Pixel 
<tb> <SEP> Downsampled 
<tb> <SEP> Pixel 

<tb> <SEP> A <SEP> Upsampled <SEP> Pixel 

<tb> <SEP> Prediction <SEP> Pixel 

<tb> 

EMI22.1 



<tb> Symbol <SEP> Pixel 

<tb> <SEP> O <SEP> Extra <SEP> Right <SEP> and 
<tb> <SEP> Left <SEP> Pixels <SEP> for 
<tb> <SEP> Upsampling 
<tb> 

As previously described, the upsampling filters may be upsampling polyphase filters, and Table 4 gives 
characteristics of these upsampling polyphase interpolation filters. 

Table 4: 
EMI22.2 



<tb> <SEP> 3:1 <SEP> 2:1 

<tb> <SEP> Upsampling <SEP> Upsampling 

<tb> Number <SEP> of <SEP> Polyphase <SEP> Filters <SEP> 3 <SEP> 2 
<tb> Number <SEP> of <SEP> Taps <SEP> 3 <SEP> 5 

<tb> Maximum <SEP> number <SEP> of <SEP> horizontal <SEP> downsampled <SEP> pixels <SEP> 9 

<SEP> 13 

<tb> 

Next two tables, Table 5 and Table 6, show polyphase filter coefficients for the exemplary3:1 and2:1 
upsampling polyphase filters. 

Table 5: 3:1 Upsampling Filter 
EMI22.3 



<tb> <SEP> Phase <SEP> 0 <SEP> Phase <SEP> 1 <SEP> Phase <SEP> 2 

<tb> Double <SEP> Precision <SEP> -0.1638231735591 <SEP> 0.0221080691070 <SEP> 

0.3737642376078 

<tb> <SEP> 0.79 <SEP> 00589359512 <SEP> 0.9557838617858 <SEP> 0.7900589359512 
<tb> <SEP> 0.3737642376078 <SEP> 0.0221080691070 <SEP> -0.1638231735591 
<tb> Fixed <SEP> Point <SEP> (9 <SEP> bits) <SEP> -0.1640625 <SEP> (-42) <SEP> 0.0234375 
<SEP> (6) <SEP> 0.3750000 <SEP> (96) 

<tb> <SEP> 0.7890625 <SEP> (202) <SEP> 0.95703125 <SEP> (244) <SEP> I <SEP> <SEP> 
0.7890625 <SEP> (202) 
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<tb> <SEP> 0.3750000 <SEP> (96) <SEP> 0.0234375 <SEP> (6) <SEP> -0.1640625(42) <SEP> 
<tb> 

Table 6: 2:1 Upsampling Filter 
EMI23.1 



<tb> <SEP> Phase <SEP> 0 <SEP> Phase <SEP> 1 

<tb> Double <SEP> Precision <SEP> 0.01 10396839260 <SEP> -0.14333638871 13 
<tb> <SEP> 0.0283886402920 <SEP> 0.64333638871 13 
<tb> <SEP> 0.921 1 43351 5636 <SEP> 0.64333638871 1 3 
<tb> <SEP> 0.0283886402920 <SEP> -0.1433363887113 
<tb> <SEP> 0.01 10396839260 <SEP> 0.0000000000000 

<tb> Fixed <SEP> Point <SEP> (9 <SEP> bits) <SEP> 0.01718750 <SEP> (3) <SEP> -0.14453125 
<SEP>(-37) 

<tb> <SEP> 0.02734375 <SEP> (7) <SEP> 0.64453125 <SEP> (165) 
<tb> <SEP> 0.92187500 <SEP> (236) <SEP> 0.64453125 <SEP> (165) 
<tb> <SEP> 0.02734375 <SEP> (7) <SEP> -0.14453125 <SEP> (-37) 
<tb> <SEP> 0.01 71 8750 <SEP> (3) <SEP> 0.00000000 <SEP> (0) 
<tb> 

In a fixed point representation, the numbers in parenthesis of Table 5 and Table 6 are 2's complement 
representations in 9 bits with the corresponding double precision numbers are on the left. Depending upon 
the subpixel position of the motion prediction block in the downsampled reference image frame, one 
corresponding phase of the polyphase interpolation filter is used. Also, for the exemplary embodiment 
additional pixels on the left and right are used to interpolate 17 horizontal pixels in the original image 
frame. For example, in the case of 3:1 decimation, a maximum of 6 horizontally downsampled pixels are 
produced for each input macroblock. However, when upsampling, 9 horizontal pixels are used to produce 
the corresponding motion prediction block values because an upsampling filter requires more left and right 
pixels outside of the boundary for the filter to operate. Since the exemplary embodiment employs half pixel 
motion estimation, 17 pixels are needed to get 16 half pixels which are the average values of nearest two 
pixel samples. A half pixel interpolator performs the interpolation operation which provides the block of 
pixels with half-pixel resolution. Table 7A illustrates an exemplary mapping between subpixel positions 
and polyphase filter elements, and shows a number of left pixels which are needed in addition to the pixels 
in the upsampled block for the upsampling process. 

Table 7A 
EMI24.1 



<tb> <SEP> Sub <SEP> Pixel <SEP> Polyphase <SEP> No. <SEP> of <SEP> Extra <SEP> Coordinate 
<tb> <SEP> Position <SEP> Left <SEP> Pixels <SEP> Change 

<tb> 3:1 <SEP> Upsampling <SEP> 0 <SEP> <SEP> 1 <SEP> 1 <SEP> x* <SEP> <SEP> - <SEP> > 
<SEP> x <SEP> 1 <SEP> 

<tb> <SEP> 1 <SEP> <SEP> 2 <SEP> 1 <SEP> x* <SEP> - > x* <SEP> -1 <SEP> 
<tb> <SEP> 2 <SEP> 0 <SEP> 0 

<tb> 2:1 <SEP> Upsampling <SEP> 0 <SEP> 0 <SEP> 2 <SEP> x* <SEP> - > <SEP> x* <SEP> -2 
<SEP> 

<tb> <SEP> 1 <SEP> 1 <SEP> 2 <SEP> x* <SEP> - <SEP> <SEP> > <SEP> x* <SEP> -2 <SEP> 
<tb> 

Figure 3B summarizes the upsampling process which is performed for each row of an input macroblock. 
First, in step 310, the motion vector for the block of the input image frame being processed is received. At 
step312, the motion vector is translated to correspond to the downsampled reference frame in memory. 
At step 314, the scaled motion vector is used to calculate the coordinates of the desired reference image 
half macroblock stored in memory 130. At step 316 the subpixel point for the half macroblock is 
determined and the initial polyphase filter values for up sampling are then determined at step 318. The 
identified pixels for the reference half macroblock of the stored downsampled reference frame are then 
retrieved from memory 130 at step 320. 

Before the first pass at the filtering step 324, the registers of the filter may be initialized at step 322, which, 
for the exemplary embodiment includes the step of loading the registers with the initial 3 or 5 pixel values. 
Then, after the filtering step 324, the process determines, at step 326, whether ail pixels have been 
processed, which for the exemplary embodiment is 17 pixels. If all pixels have been processed, the 
upsampled block is complete. For an exemplary embodiment, a 17 by 9 pixel half macroblock is returned. 
The system returns upper or lower half macroblocks to allow for motion prediction decoding of both 
progressive scan and interlaced scan images. If all pixels have not been processed, the phase is updated 
at step 328, and the phase is checked, for the 0 value. If the phase is zero, the registers are updated for 
the next set of pixel values. 

Updating the phase at step 328 updates the phase value to 0, 1 , and 2 for the filter loop period for 
exemplary 3:1 upsampling and to 0, and 1 for the filter loop period for 2:1 upsampling. Where the left-most 
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pixel is outside of a boundary of the image picture, the first pixel value in the image picture may be 
repeated. 

For an exemplary embodiment, the upsample filtering operation may be implemented in accordance with 

the following guidelines. First, several factors may beused: 1 ) the half-pixel motion prediction operation 

averages two full pixels, and corresponding filter coefficients are also averaged to provide the half-pixel 

filter coefficient; 2) a fixed number of filter coefficients, five for example, which may be 

equivalent to the number of filter taps, may be employed regardless of the particular 

downconversion; 3) five parallel input ports may be provided to the up sampling block for 

each forward and backward lower and upper block, with five input pixels LWR(O) 

LWR(4) for each clock transition for each reference block being combined with 

corresponding filter coefficients to provide one output pixel, and 4) the sum of filter 

coefficients h(0)-h(4) combined with respective pixels LWR(0)-LWR(4) provide the 

output pixel of the sampling block. 

Filter coefficients are desirably reversed because the multiplication ordering is 

opposite to the normal ordering of filter coefficients, and it may be desirable to zero some 

coefficients. Table 7B gives exemplary coefficients for the 3:1 upsampling filter, and 

Table 7C gives exemplary coefficients for the 2:1 upsampling filter: 

Table 7B 

EMI25.1 



<tb> <SEP> Subpixel <SEP> 0 <SEP> Subpixel <SEP> 1 <SEP> Subpixel <SEP> 2 <SEP> Subpixel 

<SEP> 3 <SEP> Subpixel <SEP> 4 <SEP> Subpixel <SEP> 5 

<tb> Filter <SEP> 6 <SEP> -18 <SEP> 42 <SEP> <SEP> -21 <SEP> 96 <SEP> 51 

<tb> Coeff. <SEP> 244 <SEP> 223 <SEP> 202 <SEP> 149 <SEP> 202 <SEP> 223 

<tb> <SEP> 6 <SEP> 51 <SEP> 96 <SEP> 149 <SEP> -42 <SEP> -18 

<tb> <SEP> 0 <SEP> 0 <SEP> 0 <SEP> -21 <SEP> 0 <SEP> 0 

<tb> <SEP> 0 <SEP> 0 <SEP> 0 <SEP> 0 <SEP> 0 <SEP> 0 

<tb> Reference <SEP> x*-1 <SEP> xM <SEP> x*-1 <SEP> xM <SEP> x <SEP> x <SEP> 

<tb> <SEP> Phase <SEP> i <SEP> 01 <SEP> ) <SEP> 00 <SEP> 10 <SEP> 01 <SEP> 00 <SEP> 10 

<tb> <SEP> Half <SEP> Pixel <SEP> 0 <SEP> 1 <SEP> 0 <SEP> 1 <SEP> 0 <SEP> f <SEP> 1 

<tb> 

Table 7C 
EMI26.1 



<tb> <SEP> Subpixel <SEP> 0 <SEP> Subpixel <SEP> 1 <SEP> Subpixel <SEP> 2 <SEP> Subpixel 
<SEP> 3 

<tb> Filter <SEP> Coeff, <SEP> 3 <SEP> 2 <SEP> -37 <SEP> -17 
<tb><SEP> 7 <SEP> -15 <SEP> 165 <SEP> 86 
<tb> <SEP> 236 <SEP> 200 <SEP> 165 <SEP> 200 
<tb> <SEP> 7 <SEP> 86 <SEP> -37 <SEP> -15 
<tb> <SEP> 3 <SEP> -17 <SEP> 0 <SEP> 2 

<tb> Reference <SEP> x*-2 <SEP> x*-2 <SEP> x*- <SEP> 1 <SEP> -1 <SEP> 
<tb> Phase <SEP> 00 <SEP> 00 <SEP> 01 <SEP> 01 

<tb> Half <SEP> Pixel <SEP> 0 <SEP> 1 <SEP> ~ <SEP> <SEP> 0 <SEP> 1 
<tb> 

In Tables 7B and 7C, x is the downsampled pixel position defined in equations (1) and (2), and subpixel 
position.xS, is redefined from equation (3) as equation (3') x5 = (XH)%(2Dx) (3') 

For chrominance values of the exemplary implementation, XH is scaled by two and equations (1),(2) and 
(3') are applied. In one embodiment, phase and half pixel information (coded as two bits and one bit, 
respectively) is used by motion compensation processor 220 and half-pixel generator 228 of Figure 2B. 
For example, reference block pixels are provided as U pixels first, V pixels next, and finally Y pixels. U and 
V pixels are clocked in for 40 cycles and Y pixels are clocked in for 144 cycles. Reference blocks may be 
provided for 3:1 decimation by providing the first five pixels, repeating twice, shifting the data by one, and 
repeating until a row is finished. The same method may be used for 2:1 decimation except that it is 
repeated once rather than twice. Input pixels are repeated since decimation follows addition of the output 
from motion compensation and half-pixel generation with the residual value. Consequently, for 3:1 
decimation, two of three pixels are deleted, and dummy pixels for these pixel values do not matter. 

3) DCT Domain Filtering Employing Weighting of DCT Coefficients 

The exemplary embodiment of the present invention includes the DCT filter 216 of Figure 2A processing 
the DCT coefficients in the frequency domain, which replaces a lowpass filter in the spatial domain. There 
are several advantages in DCT domain filtering instead of spatial domain filtering for DCT coded pictures, 
such as contemplated by the MPEG or JPEG standards, Most notably, a DCT domain filter is 
computationally more efficient and requires less hardware than a spatial domain filter applied to the spatial 
pixel sample values. For example, a spatial filter having N taps may use as many as N additional 
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multiplications and additions for each spatial pixel sample value, This compares to only one additional 
multiplication in the DCT domain filter. 

The simplest DCT domain filter of the prior art is a truncation of the high frequency DCT coefficients. 
However, truncation of high frequency DCT coefficients does not result in a smooth filter and has 
drawbacks such as "ringing" near edges in the decoded picture. The DCT domain lowpass filter of the 
exemplary embodiment of the present invention is derived from a block mirror filter in the spatial domain. 
The filter coefficient values for the block mirror filter are, for example, optimized by numerical analysis in 
the spatial domain, and these values are then converted into coefficients of the 
DCT domain filter. 

Although the exemplary embodiment shows DCT domain filtering in only the horizontal direction, DCT 
domain filtering can be done in either horizontal or vertical direction or both by combining horizontal and 
vertical filters. 

4) Derivation of the DCT Domain Filter Coefficients 

One exemplary filter of the present invention is derived from two constraints: first, that the filter process 
image data on a block by block basis for each block of the image without using information from previous 
blocks of a picture; and second, that the filter reduce the visibility of block boundaries which occur when 
the filter processes boundary pixel values. 

According to the first constraint, in the DCT based compression of an MPEG image sequence, for 
example, N X N DCT coefficients yield N X N spatial pixel values. 

Consequently, the exemplary embodiment of the present invention implements a DCT domain filter which 
boundary which is caused by an insufficient number spatial pixel values beyond the boundary to fill the 
residual of the filter. That is to say, coefficient values at the edge of a block cannot be properly filtered 
because the N-tap filter has values for only N/2 taps, the remaining values are beyond the boundary of the 
block. Several methods of supplying the missing pixel values exist: 1) repeat a predetermined constant 
pixel value beyond a boundary; 2) repeat the same pixel value as the boundary pixel value; and 3) mirror 
the pixel values of the block to simulate previous and subsequent blocks of pixel values adjacent to the 
processed block. Without prior information on the contents of the previous or subsequent block, the 
mirroring method of repeating pixel values is considered as a preferred method. Therefore, one 
embodiment of the present invention employs this mirroring method for the filter and is termed a "block 
mirror filter." 

The following describes an exemplary embodiment which implements a horizontal block mirror filter that 
lowpass filters 8 input spatial pixel sample values of a block. If the size of input block is an 8 X 8 block 
matrix of pixel sample values, then a horizontal filtering can be done by applying the block mirror filter to 
each row of 8 pixel sample values. It will be apparent to one skilled in the art that the filtering process can 
be implemented by applying the filter coefficients columnwise to the block matrix, or that multidimensional 
filtering may be accomplished by filtering the rows and then filtering the columns of the block matrix. 

Figure 4 shows an exemplary correspondence between the input pixel valuesxO through x7 (group XO) 
and filter taps for an exemplary mirror filter for 8 input pixels which employs a 15 tap spatial filter 
represented by tap valuesho throughh14. The input pixels are mirrored on the left side of group XO, shown 
as group X1 , and on the right side of group XO, shown as group X2. The output pixel value of the filter is 
the sum of 15 multiplications of the filer tap coefficient values with the corresponding pixel sample values. 
Figure 4 illustrates the multiplication pairs for the first and second output pixel values. 

The following shows that the block mirror filter in the spatial domain is equivalent to DCT domain filter. The 
mirror filtering is related to a circular convolution with 2N points (N=8). 

Define the vector x' as shown in equation (4). 

x'(n)=x(n)+x(2N-l-n); 0 < =n < =2N-1 (4) 
In the case of N=8, 

x'=(xO, xl, x2, x3 t x4, x5, x6 ( x7, x7, x6, x5, x4, x3, x2, xl, xO) 

Rearranging the filter tap valuesho throughhl4, and denoting the rearranged values by IV 
h'=(h7, h8, h9, h10, hll, h12, h13, h14, 0, hO, hi, h2, h3, h4, h5, h6) 

Therefore, the mirror filtered output y(n) is a circular convolution of x'(n) and h'(n) which is given by 
equation (5). 

y(n)=x'(n) 8) h'(n) (5) 

Which is equivalent to equation (6). 
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where x'[n-k] is a circular modulo of x*(n) and 
x' [n]=x'(n) for n > =0 
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The circular convolution in the spatial domain shown in equation (5) corresponds to the scalar 

multiplication in the Discrete Fourier Transform (DFT) domain. Defining 

Y(k) as the DFT of y(n), then equation (5) becomes equation (7) in the DFT domain. 

Y (k) =X'(k) H'(k) (7) 

where X^k) and H'(k) are the DFTs of x'(n) and h'(n) respectively. 

Equations (4) through (7) are valid for a filter with a number of taps less than 2N. 

In addition, the filter is limited to be a symmetric filter with an odd number of taps, with these constraints 
H'(k) is a real number. Therefore, X'(k), the DFT of x'(n), can be weighed with a real number H'(k) in the 
DFT frequency-domain instead of 2N multiplication and 2N addition operations in the spatial domain to 
implement the filtering operation. The values of X'(k) are very closely related to the DCT coefficients of the 
original N-point x(n), because an N-point DCT of x(n) is obtained by the 2N-point DFT of x'(n) which is the 
joint sequence composed of x(n) and its mirror, x(2N-l-n). 

The following describes the derivation of the DFT coefficients of the spatial filter, 

H'(k), by assuming a symmetric filter having an odd number of taps, 2N-1 , which is h(n) =h(2N-2-n), and 

equivalents h'(n) =h'(2N-n) and h'(N) =0. Define H'(k) as in equation (8). 
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whereW2Nkfl =exp(-2itkn/(2N)}; and H'(k) = H'(2N-k). 

The inventor has determined that the 2N-point DFT of x'(n), X'(k), can be expressed by its DCT 

coefficients as shown in equation (9). 

EMI30.2 

whereas the DCT coefficient of x (n), C(k), is given by equation (10). 
EMI30.3 



and C(k)=0 elsewhere. 

The values of X'(k), the DFT coefficients of x'(n), can be expressed by C(k), the 

DCT coefficients of x'(n) by the matrix of equation (1 1 ). 

EMI30.4 

The original spatial pixel sample values, x(n), can be also obtained by IDCT (Inverse Discrete Cosine 

Transformation) shown in equation (12). 
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wherea(k)=1/2 for k=0 and 1 otherwise. 

The values of y(n) for 0 < =n < =N-1 , are obtained by IDFT of X'(k)H'(k) given in (1 3): 
EMI31.1 

The values y(n) of equation (13) are the spatial values of the IDCT of C(k)H'(k). 

Therefore, the spatial filtering can be replaced by the DCT weighting of the input frequency-domain 
coefficients representing the image block with H'(k) and then performing the IDCT of the weighted values 
to reconstruct the filtered pixel values in the spatial domain. 

One embodiment of the exemplary block mirror filtering of the present invention is derived as by the 
following steps: 1 ) a one dimensional lowpass symmetric filter is chosen with an odd number of taps, 
which is less than 2N taps; 2) the filter coefficients are increased to 2N values by padding with zero's; 3) 
the filter coefficients are rearranged so that the original middle coefficient goes to the zeroth position by a 
left circular shift; 4) the DFT coefficients of the rearranged filter coefficients are determined; 5) the DCT 
coefficients are multiplied with the real number DFT coefficients of the filter; and 6) an inverse discrete 
cosine transform (IDCT) of the filtered DCT coefficients is performed to provide a block of lowpass-filtered 
pixels prepared for decimation. 

The cutoff frequency of the lowpass filter is determined by the decimation ratio. 
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For one exemplary embodiment, the cutoff frequency iss/3 for a 3:1 decimation and 7t/2 for a 2:1 
decimation, whereTc corresponds to one-half of sampling frequency. 

A DCT domain filter in MPEG and JPEG decoders allows memory requirements to be reduced because 
the inverse quantizer and IDCT processing of blocks already exists in the decoder of the prior art, and only 
the additional scalar multiplication of DCT coefficients by the DCT domain filter is required. Therefore, a 
separate DCT domain filter block multiplication is not physically required in a particular implementation; 
another embodiment of the present invention simply combines the DCT domain filter coefficients with the 
IDCT processing coefficients and applies the combined coefficients to the IDCT operation. 

For the exemplary down conversion system of the present invention, the horizontal filtering and 
decimations of the DCT coefficients were considered; and the following are two exemplary 
implementations for: 

1. 1920H by 1080V interlace to 640 by 1080 interlace conversion (Horizontal: 1 decimation). 

2. 1280H by 720V progressive to 640 by 720 progressive conversion (Horizontal 2:1 Decimation) 
Table 8 shows the DCT block mirror filter (weighting) coefficients; in Table 8 the numbers in the 
parenthesis are 1 0 bit 2's complementary representations. The "* M of 

Table 8 indicates an out of bound value for the 1 0 bit 2's complement representation because the value is 
more than 1; however, as is known by one skilled in the art, the multiplication of the column coefficients of 
the block by the value indicated by the * can be easily implemented by adding the coefficient value to the 
coefficient multiplied by the fractional value (remainder) of the filter value. 

Table 8 
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<tb> <SEP> 3:1 <SEP> Decimation <SEP> 2:1 <SEP> Decimation 

<tb> H[0] <SEP> 1.000000000000000 <SEP> (511) <SEP> 1.0000000000000000 <SEP> (511) <SEP> 

<tb> H[1] <SEP> 0.986934590759779 <SEP> (505) <SEP> 1 .0169628157945179 <SEP> (*) 

<tb> H[2] <SEP> 0.790833583171840 <SEP> (405) <SEP> 1.0000000000000000 <SEP> (511) <SEP> 

<tb> H[3] <SEP> 0.334720213357461(171) <SEP> 0.82247656390475166 <SEP> (421) 

<tb> H[4] <SEP> -0.0323463361027473 <SEP> (-17) <SEP> 0.46728234862006007 <SEP> (239) 

<tb> H[5] <SEP> -0.0377450036954524 <SEP> (-19) <SEP> 0.10634261847436199 <SEP> (54) 

<tb> H[6] <SEP> -0.0726889747390758(37) <SEP> -0.052131780559049545 <SEP> (-27) 

<tb> H[7] <SEP> 0.00954287167337307 <SEP> (5) <SEP> -0.003489737967467715 <SEP> (-2) 

<tb> 

These horizontal DCT filter coefficients weight each column in the 8 X 8 block of 

DCT coefficients of the encoded video image. For example, the DCT coefficients of column zero are 

weighted by H[0], and the DCT coefficients of the first column are weighted by H[1] and so on. 

The above description illustrates a horizontal filter implementation using onedimensional DCTs. As is 

known in the digital signal processing art, such processing can be extended to two-dimensional systems. 

Equation (12) illustrates the IDCT for the onedimensional case, consequently, equation (12') gives the 

more general two dimensional 

IDCT: 

EMI33.1 

where 
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<SEP> 1 

<tb> C(u), <SEP> C(v) <SEP> are <SEP> fl <SEP> ::k:e 
<tb> <SEP> t <SEP> 1 <SEP> otherwise 

<tb> where f(x,y) is the spatial domain representation, x and y are spatial coordinates in the sample 
domain, and u,v are the coordinates in the transform domain. Since the coefficients C(u), C(v) are known, 
as are the values of the cosine terms, only the transform domain coefficients need to be provided for the 
processing algorithms. 

For a two-dimensional system, the input sequence is now represented as a matrix of values, each 
representing the respective coordinate in the transform domain, and the matrix may be shown to have 
sequences periodic in the column sequence with period M, and periodic in the row sequence with period 
N, N and M being integers. A twodimensional DCT can be implemented as a one dimensional DCT 
performed on the columns of the input sequence, and then a second one dimensional DCT performed on 
the rows of the DCT processed input sequence. Also, as is known in the art, a twodimensional IDCT can 
be implemented as a single process. 

Figure 5 shows an exemplary implementation of the filter for down-conversion for a two-dimensional 
system processing the horizontal and vertical components implemented as cascaded one-dimensional 
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IDCTs. As shown in Figure 5, the DCT Filter Mask 216 and 

IDCT 218 of Figure 2 may be implemented by a Vertical Processor 510, containing a 
Vertical DCT Filter 530 and a Vertical IDCT 540, and a Horizontal Processor 520, containing a horizontal 
DCT Filter and horizontal IDCT which are the same as those implemented for the vertical components. 
Since the filtering and IDCT processes are linear, the order of implementing these processes can be 
rearranged (e.g, horizontal and vertical DCT filtering first and horizontal and vertical IDCTs second, or 
vise-versa, or 

Vertical Processor 520 first and Horizontal Processor 510 (second)). 

In the particular implementation shown in Figure 5, the Vertical Processor 510 is followed by a block 
Transpose Operator 550, which switches the rows and columns of the block of vertical processed values 
provided by the Vertical Processor. This operation may be used to increase the efficiency of computation 
by preparing the block for processing by the Horizontal Processor 520. 

The encoded video block, for example an 8 X 8 block of matrix values, is received by the Vertical DCT 
filter 530, which weights each row entry of the block by the DCT filter values corresponding to the desired 
vertical decimation. Next, the Vertical 

IDCT 540 performs the inverse DCT for the vertical components of the block. As described previously, 
since both processes simply perform a matrix multiplication and addition, the DCT LPF coefficients can be 
combined with the vertical DCT coefficients for matrix multiplications and addition operations. The Vertical 
Processor 510 then provides the vertically processed blocks to the Transpose Operator 550, which 
provides the transposed block of vertically processed values to the Horizontal Processor 520. The 
Transpose Operator 550 is not necessary unless the IDCT operation is only done by row or by column. 
The Horizontal Processor 520 performs the weighting of each column entry of the block by the DCT filter 
values corresponding to the desired horizontal filtering, and then performs the inverse DCT for the 
horizontal components of the block. 

As described with reference to equation (12'), only coefficients in the transform domain are provided to the 
processing algorithms; and the operations are linear which allows mathematical operations on these 
coefficients only. The operations for the IDCT, as is readily apparent from equation (12'), form a sum of 
products. Consequently, a hardware implementation requires known coefficients to be stored in memory, 
such as a 

ROM (not shown), and a group of multiply and add circuits (not shown) which receives these coefficients 
from the ROM as well as selected coefficients from the matrix of input transform coordinates. For more 
advanced systems, a ROM-accumulator method may be used if the order of mathematical operations is 
modified according to distributed arithmetic to convert from a sum of products implementation to a bit- 
serial implementation. Such techniques are described in, for example, Stanley A. White, 
Applications of Distributed Arithmetic to Digital Signal Processing: A Tutorial Review, 
IEEE ASSP Magazine, July, 1989, which take advantage of symmetries in the computations to reduce a 
total gate count of the sum of products implementation. 

In an alternative embodiment of the present invention, the DCT filter operation may be combined with the 
inverse DCT (IDCT) operation. For such an embodiment, since the filtering and inverse transform 
operations are linear, the filter coefficients may be combined with the coefficients of the IDCT to form a 
modified IDCT. As is known in the art, the modified IDCT, and hence the combined IDCT and DCT 
downconversion filtering, may be performed through a hardware implementation similar to that of the 
simple IDCT operation. 

c) Memorv Subsystem 1 ) Memory Access and Storage of Bitstream and Picture Data 
As shown in Figure 1B, the exemplary embodiment of the present invention employs an ATV Video 
Decoder 121 having a Memory Subsystem 174 which controls the storage and reading of information to 
and from Memory 130. Memory Subsystem 174 provides picture data and bitstream data to Memory 130 
for video decoding operations, and in the preferred embodiment, at least 2 pictures, or frames, are used 
for proper decoding of MPEG-2 encoded video data. An optional On-Screen Display (OSD) section in the 
Memory 130 may be available to support OSD data. The interface between the 

Memory Subsystem 174 and Memory 130 may be a Concurrent RDRAM interface providing a 500 Mbps 
channel, and three RAMBUS channels may be used to support the necessary bandwidth. An embodiment 
of the present invention having Picture processor 171, Macroblock decoder 172, and Memory subsystem 
174 operating with external memory 130 may employ a system as described in U.S. Patent 5,623,311 
entitled MPEG 

VIDEO DECODER HAVING A HIGH BANDWIDTH MEMORY to Phillips et al. f which is incorporated 
herein by reference. Figure 12 is a high level block diagram of such system of a video decoder having high 
bandwidth memory as employed by an exemplary embodiment of the present invention to 
decodeMP&commat;ML MPEG-2 pictures. 

In summary, and described with relation to Figure 1A and Figure 1B, U.S. Patent 5,623,31 1 describes a 
single, high bandwidth memory having a single memory port. The memory 130 holds input bitstream, first 
and second reference frames used for motion compensated processing, and image data representing the 
field currently being decoded. 
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The decoder includes 1 ) circuitry (picture processor 1 71 ) which stores and fetches the bitstream data, 2) 
circuitry that fetches the reference frame data and stores the image data for the currently decoded field in 
block format (Macroblock decoder 172), and fetches the image data for conversion to raster-scan format 
(display section 173). The memory operations are time division multiplexed using a single common 
memory port with a defined memory access time period, called Macroblock Time (MblkT) for control 
operations. A digital phase locked loop (DPLL) 122 counts pulses of a 27 MHz system clock signal, 
defined in the MPEG-2 standard, to generate a count value. The count value is compared to a succession 
of externally supplied system clock reference (SCR) values to generate a phase difference signal that is 
used to adjust the frequency of the signal produced by the digital phase locked loop. 

Table 9 summarizes the picture storage requirements for DC configurations to support multiple formats: 

Table 9 
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<tb> <SEP> Format <SEP> Pixels <SEP> Macro- <SEP> Pixels <SEP> Macro <SEP> Bits <SEP> per 
<SEP> Storage 

<tb> <SEP> (H) <SEP> blocks <SEP> (H) <SEP> (V) <SEP> blocks <SEP> (V) <SEP> Picture <SEP> (3 
<SEP> Pictures) 

<tb> <SEP> 1920 <SEP> x <SEP> L <SEP> 640 <SEP> 40 <SEP> 1088 <SEP> 68 <SEP> 8,355,840 
<SEP> 25,067,520 
<tb> 1088 <SEP> DC 

<tb> <SEP> 1280X <SEP> 640 <SEP> 40 <SEP> 720 <SEP> 45 <SEP> 5,529,600 <SEP> 16,588,800 
<tb> <SEP> 720 <SEP> DC 

<tb> 704x480 <SEP> 704 <SEP> 44 <SEP> 480 <SEP> <SEP> 30 <SEP> 4,055,040 <SEP> 12,165,120 
<SEP> 

<tb> 640 <SEP> x <SEP> 480 <SEP> 640 <SEP> 40 <SEP> 480 <SEP> 30 <SEP> 3,686,400 <SEP> j 

<SEP><SEP> 12,165,120 

<tb> 

For DC mode, 1920 x 1080 pictures are reduced by a factor of 3 horizontally, yielding a 640 x 1080 
picture; 1280 x 720 pictures are reduced by a factor of 2 horizontally yielding a 640 x 720 picture. The 704 
x 480 and 640 x 480 pictures are not required to be reduced in DC mode. 

Accommodating multiple DC pictures in Memory 130 also requires supporting respective decoding 
operations according to corresponding picture display timing. For example, progressive pictures occur at 
twice the rate of interlaced pictures (60 or 59.94 

Hz progressive vs. 30 or 29.97 Hz interlace) and, as a result, progressive pictures are 
decoded faster than interlaced pictures (60 or 59.94 Frames per second progressive vs. 30 
or 29.97 Frames per second interlace). Consequently, the decoding rate is constrained by 
the display rate for the format, and if the less stringent 59.97 or 29.97 frames per second 
decoding rates are used rather than the 60 or 30 frames per second, one frame out of 
every 1001 frames may be dropped from the conversion. For convenience, decoding 
operations for a format may be measured in units of "Macroblock Time" (MblkT) defined 
as the period during which all decoding operations for a macroblock may be completed 
(clock cycles per macroblock decoding). Using this period as a measure, as defined in 
equation 14, control signals and memory access operations can be defined during the 
regularly occurring MblkT period. 

MblkT (clock cycles/macroblock) = system clock rate (clock cycles/sec.) / Frame 
rate (frames/sec.) / Picture Size (macroblocks/frame) (14) 

In addition, a blanking interval may not be used for picture decoding of interlaced pictures, and an 8-line 
margin to the time period is added to account for decoding 8 lines simultaneously (interlaced) and 16 lines 
simultaneously (progressive). Therefore, an adjustment factor (AdjFact) may be added to the MblkT, as 
given in equations (15) and (16). 

AdjFact (interlace) = (total lines - vertical blank lines-8)/total lines (15) AdjFact (progressive) = (total lines - 

16)/ 16)/total lines 

(16) 

Table 10 lists MblkT for each of the supported formats: 

Table 10 
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<tb> <SEP> Format <SEP> Mblk <SEP> per <SEP> Frame <SEP> Time <SEP> MblkT <SEP> 
Adjustment <SEP> Active 

<tb> <SEP> frame <SEP> (msec) <SEP> (elks) <SEP> factor <SEP> I <SEP> Decoding 
<tb> <SEP> MblkT 

<tb> 1920 <SEP> x <SEP> 1080 <SEP> 8160 <SEP> 33.33 <SEP> 255.3 <SEP> 0.9729 <SEP> 248.4 
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<tb> <SEP> 1280 <SEP> x <SEP> 720 <SEP> 3600 <SEP> 16.67 <SEP> 289.4 <SEP> 0.9787 <SEP> 
283.2 

<tb>704 <SEP> x <SEP> 480 <SEP> P <SEP> 1320 <SEP> 16.67 <SEP> 789.1 <SEP> 0.9695 <SEP> 
765.1 

<tb>704 <SEP> x <SEP> 480 <SEP> I <SEP> 1320 <SEP> 33.33 <SEP> 1578 <SEP> 0.9419 <SEP> 
1486.6 

<tb>640 <SEP> x <SEP> 480 <SEP> P <SEP> 1200 <SEP> 16.67 <SEP> 868 <SEP> 0.9695 <SEP> 
841 .6 

<tb>640 <SEP> x <SEP> 480 <SEP> I <SEP> 1200 <SEP> 33.33 <SEP> 1736 <SEP> 0.9419 <SEP> 

1635.3 

<tb> 

In an exemplary embodiment of the present invention, a MblkT of 241 clocks is employed for all formats to 
meet the requirement of the fastest decode time including a small margin. For such chosen MblkT period, 
slower format decoding includes periods in which no decoding activities occur; consequently, a counter 
may be employed to reflect the linear decoding rate with a stall generated to stop decoding in selected 
MblkT intervals. 

Referring to Figure 1B, the Memory Subsystem 174 may provide internal picture data interfaces to the 
Macroblock decoder 172 and display section 173. A decoded macroblock interface accepts decoded 
macroblock data and stores it in correct memory address locations of Memory 130 according to a memory 
map defined for the given format. Memory addresses may be derived from the macroblock number and 
picture number. The macroblocks may be received as a macroblock row on three channels, one channel 
per 16 Mb memory device (131-136 of FigurelA) at the system clock rate. 

Each memory device may have two partitions for each picture, each partition using an upper and lower 
address. For interlaced pictures, the one partition carries Field 0 data and the other partition carries Field 1 
data, and for progressive pictures, both upper and lower partitions are treated as a single partition and 
carry data for the entire frame. Every macroblock is decoded and stored for every picture, except for 3:2 
pull down mode where decoding is paused for an entire field time period. In 3:2 pulldown mode, a signal 
having a frame rate of 24 frames per second is displayed at 60 frames (or fields) per second by displaying 
one frame twice and the next frame three times. 

A reference macroblock interface supplies stored, previously decoded picture data to the macroblock 
decoder 172 for motion compensation. The interface may supply two, one or no macroblocks 
corresponding to bi-directional predictive (B) encoding, unidirectional predictive (P) encoding or intra (I) 
encoding. Each reference block is supplied using two channels, and each channel contains one-half of a 
macroblock. For DC mode employing a decimation factor of 2, each retrieved half macroblock is1 4x9 (Y), 
10x5 (CR) and 10x5 (CB) to allow for up-sampling and half-pixel resolution. 

A display interface provides retrieved pixel data to the display section 173, multiplexing Y, CR, and CB 
pixel data on a single channel. Two display channels may be provided to support conversion from/to - 
interlaced to/from progressive formats. In DC mode, a first channel may provide up to 4 lines of interlaced 
or progressive data simultaneously and a second channel may provide up to 4 lines of interlaced data. 

For downconversion, downsampled macroblocks are merged into a single macroblock for storage. The 
downsampling process of the DC mode is described subsequently with reference to Figure 6A and Figure 
6B. Figure 6C illustrates a merging process of two macroblocks into a single macroblock for storage in 
memory 130 for downconversion by 2 horizontally. Figure 6D illustrates a merging process of three 
macroblocks into a single macroblock for storage in memory 130 for downconversion by 3 horizontally. 

d)Downsampling and Display Conversion of the Display Section 1) Down Sampling for Low Resolution 
Formats 

Down sampling is accomplished by the Down Sampling process 232 of Figure 2B to reduce the number of 
pixels in the downconverted image. Figure 6A shows the input and decimated output pixels for a 4:2:0 
signal format for 3:1 decimation. Figure 6B shows the input and decimated output pixels for 4:2:0 
chrominance type 2:1 decimation. 

Table 1 1 gives the legend identification for the Luminance and Chrominance pixels of 
Figure 6A and Figure 6B. The pixel positions before and after the down conversion of 
Figures 6A and 6B are the interlaced (3:1 decimation) and progressive (2:1 decimation) cases 
respectively. 

Table 1 1 
EMI40.1 



<tb> Symbol <SEP> Pixel 

<tb> <SEP> + <SEP> Luminance <SEP> Before 

<tb> <SEP> Decimation 
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<tb> <SEP> x <SEP> Chrominance 

<tb> <SEP> Before <SEP> Decimation 

<tb> <SEP> Luminance <SEP> After 

<tb> <SEP> decimation 

<tb> <SEP> Chrominance <SEP> After 

<tb> <SEP> Decimation 

<tb> 

For down sampling of the interlaced image, which may be the conversion from a 1920 by 1080 pixel image 
to a 640 by 1080 pixel horizontally compressed image, two out of every three pixels are decimated on the 
horizontal axis. For the exemplary 3:1 decimation, there are three different macroblock types after the 
down conversion process. 

In Figure 6A, original macroblocks were denoted by MB0, MB1, MB2. The down sampled luminance pixels 
in MB0 start at the first pixel in the original macroblock, but in 

MB1 and MB2 the down-sampled pixels start at the third and the second pixels. Also the number of down- 
sampled pixels in each macroblock are not the same. In MB0, there are 6 down-sampled pixels 
horizontally, but 5 pixels in MB1 and MB2. These three MB types are repeating, therefore Modulo 3 
arithmetic is to be applied. Table 12 summarizes the number of downsampling pixels and offsets for each 
input macroblock MB0, MB1 , 
MB2. 

Table 12: 
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<tb> <SEP> MBO <SEP> MB0 <SEP> | <SEP> MB1 <SEP> MB2 

<tb> No. <SEP> of <SEP> Down <SEP> Sampled <SEP> 6 <SEP> 5 <SEP> 5 

<tb> Luminance <SEP> Pixels 

<tb> No. <SEP> of <SEP> Down <SEP> Sampled <SEP> 3 <SEP> 3 <SEP> 2 
<tb> Chrominance <SEP> Pixels 

<tb> Offset <SEP> of <SEP> 1 st <SEP> Down <SEP> 0 <SEP> 2 <SEP> 1 
<tb> Sampled 

<tb> Luminance <SEP> Pixel 

<tb> Offset <SEP> of <SEP> 1st <SEP> Down <SEP> 0 <SEP> 1 <SEP> 2 
<tb> Sampled 

<tb> Chrominance <SEP> Pixel 
<tb> 

For downsampling of the progressive format image the luminance signal is subsampled for every second 
sample horizontally. For the chrominance signal, the downsampled pixel has a spatial position that is one- 
half pixel below the pixel position in the original image. 

2) Display Conversion 

The display section 1 73 of the ATV Decoder 1 21 of Figure 1 B is used to format the stored picture 
information (the decoded picture information) for a particular display format. Figure11A is a high level 
block diagram illustrating the display section of the 

ATV Video Decoder 121 for an exemplary embodiment of the present invention. 

Referring to Figurel 1 A, two output video signals are supported, a first output signalVIDoutl which 
supports any selected video format, and a second output signalVIDout2 which supports525l (CCIR-601 ) 
only. Each output signal is processed by separate sets of display processing elements 1 101 and 1 102, 
respectively, which perform horizontal and vertical upsampling/downsampling. This configuration may be 
preferred when the display aspect ratio does not match the aspect ratio of the input picture. An optional 
On Screen Display (OSD) section 1104 may be included to provide on screen display information to one of 
the supported output signalsVIDoUtl andVIDout2 to form display signalsVouti orVouti. All processing is 
performed at the internal clock rate except for control of the output signalsVouti orVout2 at Output 
Controllers 1 126 and 1 128, which is done at the pixel clock rate. For the preferred embodiment, the pixel 
clock rate may be at the luminance pixel rate or at twice the luminance pixel rate. 

Because the display sets of processing elements 1 101 and 1 102 operate similarly, only the operation of 
the display processing set 1 101 is described. Referring to the display processing set 1 101 , four lines of 
pixel data are provided from the memory 130 (shown In Figurel A) to the vertical processing block 282 
(shown in Figure 2B) in raster order. Each line supplies CR,Y,CB,Y data 32 bits at a time. Vertical 
Processing block 282 then filters the four lines down to one line and provides the filtered data in 32 bit 
CRYCBY format to horizontal processing block 284 (also shown in Figure 2B). The horizontal processing 
block 284 provides the correct number of pixels for the selected raster format as formatted pixel data. 
Consequently, the filtered data rate entering the horizontal processing block 284 is not necessarily equal 
to the output data rate. In an upsampling case, the input data rate will be lower than the output data rate. 
In a down sampling case, the input data rate will be higher than the output data rate. The formatted pixel 
data may have background information inserted by optional background processing block 1110. 
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As would be known to one skilled in the art, the elements of the display section 173 are controlled by a 
controller 1 150, which is set up by parameters read from and written to the microprocessor interface. The 
controller generates signal CNTRL, and such control is necessary to coordinate and to effect proper circuit 
operation, loading and transfer of pixels, and signal processing. 

Data from the horizontal processing block 284, data from a second horizontal processing block 284a, and 
HD (non processed) Video data on HD Bypass 1 122 are provided to Multiplexer 118 which selects, under 
processor control (not shown), one video data stream which is provided to mixer 1 16 to combine the video 
data stream and optional OSD data from OSD processor 1104 into mixed output video data. The mixed 
video output data is then provided to MUXs 1 120 and 1 124. 

For the first set of processing elements 1 101 , MUX 1 120 may select from mixed output video data, HD 
data provided on HD bypass 1 122, or data from background insertion blockl 110.. The selected data is 
provided to output control processor 1 126 which also receives the pixel clock. Output control processor 
1 126 then changes the data clock rate from the internal processing domain to the pixel clock rate 
according to the output mode desired. 

For the second processing elements 1 102, MUX 1 124 may select from mixed output video data or data 
from background insertion blockl 1 10a.. The selected data is provided to output control processor 1 128 
which also receives the pixel clock. Output control processor 1 128 then changes the data clock rate from 
the internal processing domain to the pixel clock rate according to the output mode desired. MUX 1 132 
provides either the received selected data (601 Data Out) of MUX 1 124 or optional OSD data from 
OSD processor 1 104. 

Raster Generation and Control processor 1 1 30 also receives the pixel clock and includes counters (not 
shown) which generate the raster space, allowing control commands to be sent on a line by line basis to 
Display Control Processor 1 140. Display 

Control processor 1 140 coordinates timing with the external memory 130 and starts the processing for 
each processing chain 1 101 and 1102 on a line by line basis synchronized with the raster lines. Processor 
1130 also generates the horizontal, vertical and field synchronization signals (H, V and F). 

Figuresl 1B through 1 1D relate the output modes provided by Display section 173 shown in Figurel I A of 
the Video Decoder 121 to the active blocks of Figure 1 A. 

Figurel 1 B illustrates a 27 MHz Dual output mode which, for which the video data is 525P or 525I, first 
processor 1 101 (shown in Figures 1 1 A) provides 525P video data to 27 MHz DAC 143 as well as525l data 
(601 Data Out) to NTSC Encoder 152. Figurel 1C illustrates that in 27 MHz single output mode, only525l 
data (601 Data Out) is provided to NTSC encoder 152. Figurel 1D illustrates a 74 MHz/27 MHz mode in 
which the output mode matches the input format and the video data is provided to either the 27 MHz DAC 
143 or 74 MHz DAC 141 depending on the output format. The 74 

MHz DAC is used for 1920 x 1088 and 1280 x 720 pictures; the 27 MHz DAC is used for all other output 
formats. 

Display conversion of the downsampled image frames is used for display the image in a particular format. 
As noted previously, the Display Conversion block 280 shown in Figure 2B includes the vertical 
processing block (VPF) 282 and horizontal processing block (H2PF) 284 which adjust the down converted 
and down sampled images for display on the lower resolution screen. 

VPF 282 which, for the exemplary embodiment, is a vertical line interpolation processor implemented as a 
programmable polyphase vertical filter, and HZPF 284 which, for the exemplary embodiment, is a 
horizontal line interpolation processor also implemented as a programmable horizontal polyphase filter. 
The filters are programmable, which is a design option in order to accommodate display conversion for a 
number of display formats. 

As shown in Figure 2B, four lines of downsampled pixel data enter the VPF 282 in raster order. For the 
exemplary embodiment this data includes luminance (Y) and chrominance (CR and CB) pixel pairs which 
enter VPF 282 32 bits at a time. VPF 282 filters the four lines of data into one line and passes this line to 
the HZPF 284 as 32 bit values each containing luminance and chrominance data in a YCRYCB, and 
HZPF 284 then generates the correct number of pixels to match the desired raster format. 

Figure 7A is a high level block diagram illustrating an exemplary filter suitable for use as the VPF 282 of 
one embodiment of the present invention. In the following, the 

VPF 282 is described as processing pairs of input pixels (each pair includes two luminance pixels, Y, pixel 
and a chrominance CR or CB, pixel) to produce a pair of output pixels. This facilitates processing of the 
4:2:0 format because color pixels may be easily associated with their corresponding luminance pixels. 
One skilled in the art, however, would realize that only luminance pixels or only chrominance pixels may 
be so processed. 
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In addition, the VPF 282 as described produces lines in progressive format. In another embodiment 
employing'a dual output and supporting both a main output channel and a secondary output channel, a 
aecond VPF 282 may be added. 

Referring to Figure 7A, VPF 282 includes a VPF Controller 702; first muliplexer network including 
Luminance Pixel MUXs (LP MUXs) 706, 708, 710, and 712 and 

Chrominance Pixel MUXs (CP MUXs) 714, 716, 718, and 720; second multiplexer network including 

Luminance Filter MUXs (LF MUXs) 726, 728, 730 and 732 and 

Chrominance Filter MUXs (CF MUXs) 734, 736, 738 and 740; Luminance Coefficient 

RAM 704; Chrominance Coefficient RAM 724; Luminance Coefficient Multipliers 742, 744, 746, and 748; 

Chrominance Coefficient Multipliers 750, 752, 754, and 756; 

Luminance Adders 760, 762 and 764; Chrominance Adders 766, 768 and 770; Round and 
Clip processors 772 and 776; Demux/Registers 774 and 778; and Output Register 780. 

The operation of the VPF 282 is now described. Vertical resampling is accomplished with two 4-Tap 
polyphase filters, one for the Luminance pixels and one for the Chrominance pixels. The following details 
operation of the filter for the Luminance pixels only, since the operation for the Chrominance pixels is 
similar, but points out those differences in the paths as they occur. Vertical filtering of Luminance pixels 
can use up to 8 phases in the 4-Tap polyphase filter and filtering of Chrominance pixels can use up to 16 
phases in the 4-Tap polyphase filter for the exemplary embodiment. The VPF 

Controller 702, at the beginning of a field or frame, resets the vertical polyphase filter, provides control 
timing to the first and second multiplexer networks, selects coefficient sets from Luminance Coefficient 
RAM 704 and Chrominance Coefficient RAM 724 for the polyphase filter phases, and includes a counter 
which counts each line of the field or frame as it is processed. 

The VPF Controller .702, in addition to coordinating the operation of the network of MUXs and the 
polyphase filters, keeps track of display lines by tracking the integer and fractional parts of the vertical 
position in the decoded picture. The integer part indicates which lines should be accessed and the 
fractional part indicates which filter phase should be used. Furthermore, use of modulo N arithmetic when 
calculating the fractional part allows less than 1 6 phases to be used, which may be efficient for exact 
downsampling ratios such as 9 to 5. The fractional part is always truncated to one of the modulo N phases 
that are being used. 



* As shown in Figure 7A, luminance and chrominance pixel pairs from the four image lines are separated 
into a chrominance path and a luminance path. The 16 bit pixel pair data in the luminance path may be 
further multiplexed into an 8-bit even (Y-even) and 8-bit odd (Y-odd) format by LP MUXs 706, 708, 710, 
and 712, and the 16 bit pixel pair in the chrominance path into an 8-bit CR and 8-bitCg format by CP 
MUXs 714, 716, 718 and 720. The luminance filter MUXs 706, 708, 710 and 712 are used to repeat pixel 
values of a line at the top and a line at the bottom at the boundaries of a decoded image in order to allow 
filter pixel boundary overlap in the polyphase filter operation. 

Pixel pairs for the four lines corresponding to luminance pixel information and chrominance pixel 
information are then passed through the respective polyphase filters. 

Coefficients used by Multipliers 742, 744, 746 and 748 for weighting of pixel values for a filter phase are 
selected by the VPF Controller 702 based on a programmed up or down sampling factor. After combining 
the weighted luminance pixel information in Adders 760, 762 and 764, the value is applied to the Round 
and Clip processor 772 which provides eight bit values (since the coefficient multiplication occurs with 
higher accuracy). DEMUX register 774 receives the first 8 bit value corresponding to an interpolated 8 bit 
even (Y-even) luminance value and second 8-bit value corresponding to the interpolated 8-bit odd (Y-odd) 
value, and provides a vertical filtered luminance pixel pair in 16 bits. Register 780 collects and provides 
the vertical filtered pixels in the luminance and chrominance paths and provides them as vertically filtered 
32 bit values containing a luminance and chrominance pixel pair. 

Figure 7B shows the spatial relationships between the coefficients and pixel sample space of the lines. 
The coefficients for the luminance and chrominance polyphase filter paths each have 40 bits allocated to 
each coefficient set, and there is one coefficient set for each phase. The coefficients are interpreted as 
fractions with a denominator of 512. 

The coefficients are placed in the 40-bit word from left to right, CO to C3. CO and C3 are signed ten bit 2's 
complement values, andC1 and C2 are 10 bits which have a given range, for example, from -256 to 767, 
which are each subsequently converted to ll-bit 2's complement values. 

Figure 7A includes an optional luminance coefficient adjustment 782 and chrominance coefficient 
adjustment 784. These coefficient adjustments 782 and 784 are used to derive the 1 1 bit 2's complement 
number ford and C2. If bits 8 and 9(the most significant bit) are both 1 , then the sign of the eleven bit 
number is 1 (negative), otherwise the value is positive. 
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Figure 8A is a high level block diagram illustrating an exemplary filter suitable for use as the HZPF 284 of 
one embodiment of the present invention. HZPF 284 receives a luminance and chrominance pixel 
information pair, which may be 32-bit data, from the 

VPD 282. The HZPF 284 includes a HZPF Controller 802; CR latches 804;Cg latches 806; Y latches 808; 
Selection MUXs 810; Horizontal Filter Coefficient RAM 812; 

Multiplying network 814; Adding network 816; Round and Clip processor 818, DEMUX register 820 and 
output register 822. 

Horizontal resampling is accomplished by employing an 8 tap, 8 phase polyphase filter. Generation of 
display pixels is coordinated by the HZPF Controller 802 by tracking the integer and fractional parts of the 
horizontal position in the decoded and downsampled picture. The integer part indicates which pixels are to 
be accessed and the fractional part indicates which filter phase should be used. Using modulo N 
arithmetic when calculating the fractional part may allow for less than 8 phases to be used. For example, 
this may be useful if an exact downsampling ratio such as 9 to 5 is used. If the down-sampling ratio cannot 
be expressed as a simple fraction, the fractional part is truncated to one of the N phases. The HZPF 284 
of the exemplary embodiment of the present invention filters pixel pairs, and uses alignment on even pixel 
boundaries to facilitate processing of the 4:2:0 formatted picture and to keep the CR andCg pixels (the 
color pixels) together with the corresponding Y pixels. 

The operation of the HZPF 284 is now described with reference to Figure 8A. 

The HZPF Controller 802, at the beginning of a horizontal line, resets the horizontal polyphase filter, 
provides control timing to the first and second multiplexer networks, selects coefficient sets from 
Horizontal Coefficient RAM 812 for the CR.CB and Y filter coefficients for each of the polyphase filter 
phases, and selects each set of CR, CB and Y values for processing. In addition, when the horizontal 
position is near the left or right side of the line, the HZPF Controller 802 forces the edge pixel values to be' 
repeated or set to 0 for use by the 8-tap polyphase filter. Any distortion in the image caused by this 
simplification is usually hidden in the overscan portion of the displayed image. 

The pixel data received from the VPF 282 is separated into Y, CR andCB values, and these values are 
individually latched into CR latches 804;Cg latches 806; and Y latches 808 for filtering. The HZPF 
Controller 802 then selects the Y, CR andCg values by an appropriate signal to the selection MUXs 810. 
In the exemplary embodiment, there are more Y values than CR orCg values so the filter uses additional 
latches in the Y luminance latches 808. At the same time, the HZPF Controller 802 selects the appropriate 
filter coefficients for the filter phase, and for the CR orCg or Y values, based on a programmed upsampling 
or downsampling value by a control signal to Horizontal 
Filter Coefficient RAM 812. 

Horizontal Filter Coefficient RAM 812 then outputs the coefficients to the respective elements of the 
Multiplying Network 814 for multiplication with the input pixel values to produce weighted pixel values, and 
the weighted pixel values are combined in 

Adding Network 816 to provide a horizontally filtered CR, CB or Y value. 

After combining the weighted pixel values in Adding network 816, the horizontally filtered pixel value is 
applied to the Round and Clip processor which provides eight-bit values (since the coefficient 
multiplication occurs with higher accuracy). DEMUX register 820 receives a series of 8 bit values 
corresponding to a CR value, an 8 bit even (Y-even) Y value, an eight-bitCg value, and finally an eight-bit 
value corresponding to an 8-bit odd (Y-odd) Y value; and the DEMUX register 820 multiplexes the values 
into a horizontally filtered luminance and chrominance pixel pair having a 32 bit value (Y even, CR, Y 
odd.CB). Register 822 stores and provides the pixel pair as a vertically and horizontally filtered 32 bit pixel 
luminance and chrominance pixel pair. 

Figure 8B illustrates the spatial relationships between coefficients stored in 

Horizontal Filter Coefficient RAM 812 and used in the polyphase filter and the pixel sample values of the 
down sampled image for a horizontal line. The coefficients for the exemplary embodiment are placed in a 
64 bit word from left to right, CO to 01. The coefficients CO, C1 , C6 and C7 are signed 7-bit 2*s 
complement values, and C2 andC5 are signed 8-bit 2's complement and C3 and C4 are signed 10-bit 2*s 
complement values representing a range from -256 to 767. C3 and C4 are adjusted to derive the 1 1-bit 2*s 
complement values.. If both bit 8 and bit 9 (the most significant bit) are 1 , then the sign of the 1 1-bit value 
is 1 (negative), otherwise the value isO (positive). All coefficients can be interpreted as fractions with a 
denominator of 512. 

Table 13 lists coefficient for the VPF 282 and HZPF 284 for exemplary embodiments of the present 
invention performing the indicated format conversion. 

Table 13 

Coefficients for 750P to 525P or 750P to 5251 
4 tap and 2 polyphase Luminance Vertical Filter 
EMI49.1 
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<tb> <SEP> Tap <SEP> 0 <SEP> Tap <SEP> 1 <SEP> Tap <SEP> 2 <SEP> Tap <SEP> 3 
<tb> Phase <SEP> 0 <SEP> 103 <SEP> 306 <SEP> 103 <SEP> 0 
<tb> Phase <SEP> 1 <SEP> 10 <SEP> 246 <SEP> 246 <SEP> 10 
<tb> 

Coefficients for 750P to 525P or 750P to 5251 4 tap and 4 polyphase Chrominance Vertical Filter 
EMI49.2 



<tb> <SEP> Tap <SEP> 0 <SEP> Tap1 <SEP> Tap <SEP> 2 <SEP> Tap <SEP> 3 <SEP> 

<tb> Phase <SEP> 0 <SEP> 25 <SEP> 462 <SEP> 25 <SEP> 0 

<tb> Phase <SEP> 1 <SEP> -33 <SEP> 424 <SEP> j <SEP> 145 <SEP> -24 

<tb> Phase <SEP> 2 <SEP> -40 <SEP> 296 <SEP> 296 <SEP> 40 

<tb> Phase <SEP> 3 <SEP> -24 <SEP> 145 <SEP> 424 <SEP> -33 

<tb> 

Coefficients for 750P to 5251 4 tap and 2 polyphase Luminance Vertical Filter 
EMI50.1 



<tb> <SEP> Tap <SEP> 0 <SEP> Tap <SEP> 1 <SEP> Tap <SEP> 2 <SEP> Tap <SEP> 3 
<tb> Phase <SEP> 0 <SEP> 145 <SEP> 222 <SEP> 145 <SEP> 0 
<tb> Phase <SEP> 1 <SEP> 84 <SEP> 172 <SEP> 172 <SEP> 84 
<tb> 

Coefficients for 750P to 5251 4 tap and 4 polyphase Chrominance Vertical Filter 
EMI50.2 



! <tb> <SEP> Tap <SEP> 0 <SEP> Tap <SEP> 1 <SEP> Tap <SEP> 2 <SEP> Tap <SEP> 3 
<tb> Phase <SEP> 0 <SEP> 57 <SEP> 398 <SEP> 57 <SEP> 0 
<tb> Phase <SEP> 1 <SEP> -6 <SEP> 382 <SEP> 166 <SEP> -30 
<tb> Phase <SEP> 2 <SEP> -29 <SEP> 285 <SEP> 285 <SEP> -29 
<tb> Phase <SEP> 3 <SEP> -30 <SEP> 166 <SEP> 382 <SEP> -6 
<tb> 

Coefficients for 11251 to 525P 4 tap and 8 polyphase Luminance Vertical Filter 
EMI50.3 



<tb> <SEP> Tap <SEP> 0 <SEP> Tap <SEP> 1 <SEP> Tap <SEP> 2 <SEP> Tap <SEP> 3 

<tb> Phase <SEP> 0 <SEP> 20 <SEP> 472 <SEP> 20 <SEP> 0 

<tb> Phase <SEP> 1 <SEP> -20 <SEP> 425 <SEP> 70 <SEP> 37 

<tb> Phase <SEP> 2 <SEP> -52 <SEP> 472 <SEP> 161 <SEP> -69 

<tb> Phase <SEP> 3 <SEP> -62 <SEP> 397 <SEP> 238 <SEP> -61 

<tb> Phase <SEP> 4 <SEP> -63 <SEP> 319 <SEP> 319 <SEP> -63 

<tb>Phase <SEP> 5 <SEP> -61 <SEP> 238 <SEP> 397 <SEP> -62 <SEP> 

<tb> Phase <SEP> 6 <SEP> -69 <SEP> 161 <SEP> 472 <SEP> -52 

<tb> Phase <SEP> 7 <SEP> 37 <SEP> 70 <SEP> 425 <SEP> -20 

<tb> 

Coefficients for 11251 to 525P 4 tap and 16 polyphase Chrominance Vertical Filter 
EMI50.4 



<tb> <SEP> Tap <SEP> 0 <SEP> Tap <SEP> 1 <SEP> Tap <SEP> 2 <SEP> Tap <SEP> 3 

<tb> Phase <SEP> 0 <SEP> 29 <SEP> 454 <SEP> 29 <SEP> 0 

<tb> Phase <SEP> 1 <SEP> 13 <SEP> 455 <SEP> 49 <SEP> -5 

<tb> Phase <SEP> 2 <SEP> 0 <SEP> 445 <SEP> 73 <SEP> -6 

<tb> Phase <SEP> 3 <SEP> -9 <SEP> 428 <SEP> 101 <SEP> -8 

<tb> 

EMI51.1 



<tb> <SEP> Tap <SEP> 0 <SEP> Tap <SEP> 1 <SEP> Tap <SEP> 2 <SEP> Tap <SEP> 3 

<tb> Phase <SEP> 4 <SEP> -15 <SEP> 404 <SEP> 132 <SEP> -9 

<tb> Phase <SEP> 5 <SEP> -18 <SEP> 376 <SEP> 165 <SEP> -1 1 

<tb> Phase <SEP> 6 <SEP> -20 <SEP> 345 <SEP> 201 <SEP> -14 

<tb> Phase <SEP> 7 <SEP> -19 <SEP> 310 <SEP> 237 <SEP> -16 

<tb> Phase <SEP> 8 <SEP> -18 <SEP> 274 <SEP> 274 <SEP> -18 

<tb> Phase <SEP> 9 <SEP> -16 <SEP> 237 <SEP> 310 <SEP> -19 

<tb> Phase <SEP> 10 <SEP> -14 <SEP> 201 <SEP> 345 <SEP> -20 
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<tb> Phase <SEP> 1 1 <SEP> -1 1 <SEP> 1 65 <SEP> 376 <SEP> -1 8 
<tb> Phase <SEP> 12 <SEP> -9 <SEP> 132 <SEP> 404 <SEP> -15 
<ib> Phase <SEP> 13 <SEP> -8 <SEP> 101 <SEP> 428 <SEP> -9 
<tb> Phase <SEP> 14 <SEP> -6 <SEP> 73 <SEP> 445 <SEP> 0 
<tb> Phase <SEP> 15 <SEP> -5 <SEP> 49 <SEP> 455 <SEP> 13 
<tb> 

In the exemplary embodiments of the display conversion system horizontal conversion is, in part 
performed by the DCT domain filter 21 6, and the downsampling 

Processor 232 shown in Figure 2B. These provide the same number of horizontal pixels (640) whether the 
conversion is from 11251 or 750P. Accordingly, the HZPF 284 upsamples these signals to provide 720 
active pixels per line and passes 525P or 5251 signals unmodified, as these signals have 720 active pixels 
per line as set forth above in 

Tables 1 and 2, the values of the coefficients of the Horizontal Filter do not change for conversion to 
480P/480I/525P/525I. These Horizontal filter coefficients are given in 
Table 14. 

Table 14 

Coefficients for Horizontal Filter 
EMI51.2 



<tb> <SEP> Tap <SEP> Tap <SEP> 1 <SEP> <SEP> Tap <SEP> 2 <SEP> <SEP> Tap <SEP> 3 <SEP> 

Tap <SEP> 4 <SEP> Tap <SEP> 5 <SEP> Tap <SEP> 6 <SEP> Tap <SEP> 7 

<tb> Phase <SEP> 0 <SEP> -8 <SEP> 13 <SEP> -17 <SEP> 536 <SEP> -17 <SEP> 13 <SEP> -8 

<SEP>0 

<tb> Phase <SEP> 1 <SEP> -13 <SEP> 28 <SEP> -62 <SEP> 503 <SEP> 48 <SEP> -9 <SEP> 0 
<SEP>17 

<tb> Phase <SEP> 2 <SEP> -14 <SEP> 37 <SEP> -90 <SEP> 477 <SEP> 134 <SEP> -37 <SEP> 10 
<SEP> -5 

<tb> Phase <SEP> 3 <SEP> -13 <SEP> 38 <SEP> -96 <SEP> 406 <SEP> 226 <SEP> -64 <SEP> 22 
<SEP> -7 

<tb> Phase <SEP> 4 <SEP> -10 <SEP> 31 <SEP> -85 <SEP> 320 <SEP> 320 <SEP> -85 <SEP> 31 
<SEP> -10 

<tb> Phase <SEP> 5 <SEP> -7 <SEP> 22 <SEP> -64 <SEP> 226 <SEP> 406 <SEP> -96 <SEP> 38 

<SEP>-13 

<tb> 

EMI52.1 



<tb> <SEP> Tap <SEP> 0 <SEP> Tap <SEP> Tap <SEP> 2 <SEP> <SEP> Tap3 <SEP> Tap4 <SEP> 
TapS <SEP> Tap6 <SEP> Tap7 <SEP> 

<tb> Phase <SEP> 6 <SEP> -5 <SEP> 10 <SEP> -37 <SEP> 2 <SEP> <SEP> 134 <SEP> 3 <SEP> 

<SEP> 477 <SEP> 4 <SEP> <SEP> -90 <SEP> 5 <SEP> <SEP> 37 <SEP> -14 

<tb>Phase7 <SEP> 17 <SEP> O <SEP> -9 <SEP> 48 <SEP> 503 <SEP> -62 <SEP> 28 <SEP> -13 

<tb> 

In addition, the programmable capability of the HZPF 284 allows for a nonlinear horizontal scan. Figure 9A 
illustrates a resampling ratio profile which may be employed with the present invention. As shown the 
resampling ratio of the HZPF 284 may be varied across the horizontal scan line and may be changed in 
piecewise linear fashion. In the exemplary configuration of Figure 9A, at the beginning of the scan line, the 
resampling ratio increases (or decreases) linearly until a first point on the scan line, where the resampling 
ration is held constant until a second point is reached where the resampling ratio decreases (or increases) 
linearly. Referring to Figure 9A, h-initial-resampling ratio is the initial resampling ratio for a picture, h 
resampling-ratio change is the first change per pixel in the resampling ratio,-h resampling-ratio change is 
the second change per pixel in the resampling ratio, andh resampling-ratio-hold column 
andh-resamplingratio-reverse-column are the display column pixel points between which the resampling 
ratio is held constant. The value display-width is the last pixel (column) of the picture line. 

Figures 9B and 9C show ratio profiles for mapping a 4:3 picture onto a 16:9 display. The ratios are defined 
in terms of input value to output value, so 4/3 is downsampling by 4 to 3 and 1/3 is up sampling 1 to 3. The 
ratio profiles shown in 

Figures 9B and 9C map an input picture image having 720 active pixels to a display having 720 active 
pixels. For example, in Figure 9B mapping a 4:3 aspect ratio display to a 16x9 aspect ratio display uses a 
4/3 downsampling, but to fill all the samples of the display requires a 1/1 average across the horizontal 
line. Consequently, the profile of 

Figure 9B has the correct aspect ratio in the center between display pixels 240 and 480, while the values 
at the sides are upsampled to fill the display. Figures 9D and 9E illustrate the profiles used for resizing 
from a 16x9 display image to a 4:3 display which is the inverse of the profiles shown in Figures 9B and 9C. 

The effect of using resampling ratio profiles according to an exemplary embodiment of the present 
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invention may be seen pictorially in Figure 10. A video transmission format having either a1 6x9 or 4x3 
aspect ratio may be displayed as either 16x9 or 4x3, but the original video picture may be adjusted to fit 
within the display area. 

Consequently, the original video picture may be shown in full, zoom, squeeze, or variable expand/shrink. 

The system allows users to select a preferred mapping between the aspect ratio of the received video 
signal and the aspect ratio of the display device, when these aspect ratios are incompatible. As set forth 
above, the control processor 207 (shown in Figure 2A) receives the aspect ratio of the received image 
from the parser 209. The control processor 207 also determines the aspect ratio of the display device (not 
shown) which is connected to receive the output signal of the system, if, for example, the display device is 
connected to the S-video output 153 or the composite video output 154 (both shown in 
Figurel A), then the aspect ratio of the display device must be 4 by 3. If, however, the display device is 
connected to the primary video output port 146, the aspect ratio may be either 4 by 3 or 16 by 9. 

In the exemplary embodiment of the invention, the user specifies the aspect ratio of the display device as 
a part of a start-up process which may be invoked through the remote control IR receiver 208 (shown in 
Figure 2A). The start up process may only be run if the video decoder system includes a primary output 
port and the system senses that there is a display device coupled to the primary output port. The start-up 
process may determine the display format of the display device (i.e. aspect ratio and maximum video 
resolution) in several ways. First, the process may present the user with a menu of possible display 
devices, each represented, for example, by a manufacturer name and model number. The user may then 
use the remote control device to select one of these display devices. The decoder system may be 
configured with a modem to periodically contact a central location to receive an updated list of display 
types as well as other updates to the programming of the controller 207. Alternatively, this type of 
information may be encoded in the user data of a received ATSC video signal and the decoder may be 
programmed to access this information to update its internal programming. 

Alternatively, to determine the aspect ratio of the display device, the user may be presented with a menu 
showing a 4 by 3 rectangle and a 16 by 9 rectangle and asked to indicate which is more like their display 
device. As another alternative, the user may asked to choose two menu choices, one listing possible video 
display resolutions and another listing possible aspect ratios. 

As yet another alternative, the control processor 207 may program the on-screen display generator to 
produce a figure, for example a circle, in several different signal resolutions (e.g. 525I, 525P.750P, 11801 
and 1 180P) and several different aspect ratios (e.g. 4 by 3 and 16 by 9), with text asking the viewer to 
press a button on the remote control device (not shown) when the best circle is displayed. The system 
may sequentially provide each of these images for a few seconds at the primary output to correlate the 
pressing of the button on the remote control device with the display of a particular image. This will provide 
the system with the needed information on image resolution and aspect ratio for the display device. 

With information on the display format of the display device, the system may automatically adapt the 
received video signal for the best possible presentation on the display device. When, for example, there is 
a mismatch between the aspect ratio of the received video signal and the aspect ratio of the display 
device, this may be indicated to the viewer and the viewer may be allowed, by invoking a command using 
the remote control device (not shown), to sequentially see all of the possible conversions between the two 
aspect ratios, as shown in Figures 9A through 9E and Figure 10, and to select one of these conversions to 
be used. This applies when the aspect ratio of the received video signal is 4 by 3 and the aspect ratio of 
the display device is 16 by 9 as well as when the aspect ratio of the received video signal is 16 by 9 and 
the aspect ratio of the display device is 4 by 3. 

As a final alternative, the system may be configured to sense information provided by the display device in 
order to determine the display format. For example, a two-way data path may be provided via one of the 
output signal lines (Y, CR, CB) of the decoder system by which data in a digital register in the display 
device may be read. The data in this register may indicate a manufacturer and model number or a 
maximum resolution and aspect ratio for the display device. Alternatively, the display device may impose a 
direct current (DC) signal on one or more of these lines and this signal may be sensed by the decoder 
system as an indication of the display format of the display device. 

It is contemplated that a multi-sync monitor, which is capable of displaying video signals having several 
different display formats may be connected to the primary output port of the video decoder. In this 
instance, the video resolution component of the display type information recovered by the control 
processor 207 desirably includes an indication that the display is a multi-sync device so that the only 
format conversion that occurs is the aspect ratio adaptation shown in Figures 9A through 9E and Figure 
10, when the aspect ratio of the received video signal does not match that of the display device. 

While exemplary embodiments of the invention have been shown and described herein, it will be 
understood that such embodiments are provided by way of example only. Numerous variations, changes, 
and substitutions will occur to those skilled in the art without departing from the spirit of the invention. 
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Accordingly, it is intended that the appended claims cover all such variations as fall within the scope of the 
invention. 
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MPEG DECODER PROVIDING MULTIPLE STANDARD OUTPUT SIGNALS 

Claims of correspondent: WO9841011 



What is Claimed: 

1 . A digital video signal conversion system for decoding and reformatting a baseband encoded digital 
video signal which sequentially represents first and second video display formats to produce a decoded 
video signal suitable for display in a predetermined video display format, each video display format having 
an aspect ratio and a resolution, the system comprising: 

a digital video signal decoder including: 

an input terminal coupled to receive the encoded digital video signal; 
data retrieval circuitry which extracts from the encoded digital video signal 
an indication of the video display format for the decoded video signal; and 
signal processing circuitry which decodes the encoded digital video signal 
to produce an output video signal, wherein the decoder is responsive to a control 
signal to process the encoded digital video signal according to respective first and 
second down conversion processes to produce an output signal in the 
predetermined video display format from encoded digital video signals in the 
respective first and second video display formats; and 

a controller, coupled to receive data which identifies the extracted video display format, the controller 
producing control signals for the digital video signal decoder which cause the decoder to convert the 
encoded digital video signal corresponding to the extracted display format into the output video signal 
having the predetermined video display format in both the first and second intervals. 

2. A digital video signal conversion system according to claim 1, wherein the encoded digital video signal 
is encoded using a frequency-domain transform operation and the digital video signal decoder includes a 
low-pass filter which operates on the frequency-domain transformed digital video signal to cause the 
digital video signal decoder to produce a decoded digital video signal having a resolution less than a 
resolution specified by the extracted video display format. 

3. A digital video signal conversion system according to claim 2, wherein the digital video signal decoder 
includes a programmable spatial filter which is responsive to a format conversion control signal provided 
by the controller to resample the decoded digital video signal provided by the digital video decoder to 
produce the output video signal having the predetermined video display format. 

4. A digital video signal conversion system according to claim 3, wherein the digital video signal is 
encoded using an encoding technique specified by the moving pictures experts group (MPEG) in which 
the encoded digital video signal is contained in a plurality of packetized elementary stream (PES) packets, 
each PES packet having a header, and the data retrieval circuitry of the digital video signal decoder 
includes means for extracting the video display format of the encoded digital video signal from the header 
of one of the PES packets. 

5. A digital video signal conversion system according to claim 3, wherein the digital video signal is 
encoded using an encoding technique specified by the moving pictures experts group (MPEG) in which 
the encoded digital video signal is contained in a digital bit-stream including sequence records, each 
sequence record having a header, and the data retrieval circuitry of the digital video signal decoder 
includes means for extracting the video display format of the encoded digital video signal from the header 
of one of the sequence record. 

6. A digital video signal conversion system according to claim 1 further including a user input device 
through which a user may enter the data which identifies the predetermined video display format. 

7. A digital video signal conversion system according to claim 1 wherein the video display device includes 
a register which holds an identification signal and the controller is coupled to the display device to read the 
identification signal and, use the identification signal to generate the data identifying the predetermined 
video display format. 

8. A digital video signal conversion system according to claim 1, wherein the digital video signal decoder 
further includes: 

analog to digital conversion means for converting the output signal of the signal processing circuitry of the 
video decoder to an analog signal; 

transmission means for applying the analog signal to the display device; 

means, coupled to the controller, for sensing a direct current (DC) potential applied to the transmission 
means by the display device; 

wherein the controller, responsive to the means for sensing, identifies the predetermined display type to 
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be used by the digital video signal decoder. 

9. A digital video signal conversion system according to claim 6 wherein: 

the video signal decoder further includes an on-screen display processor which may be programmed to 
produce a plurality of output video signals, each corresponding to a respectively different one of the 
plurality of video display formats; and 

the controller includes means, responsive to a first user control signal provided via the user input device, 
to cause the on-screen display device to sequentially provide video signals having respectively different 
video display formats and responsive to a second selection signal provided via the user input device to 
define one of respective video display formats as the predetermined video display format. 

10. A digital video signal conversion system for decoding and reformatting a baseband encoded digital 
video signal, which is encoded using a frequency-domain transform operation and which may represent a 
plurality of video display formats, to produce a decoded video signal, each video display format having an 
aspect ratio and a resolution, the system comprising: 

a display device having a predetermined video display format; 
a digital video signal decoder including: 

an input terminal coupled to receive the encoded digital video signal; 

data retrieval circuitry which extracts from the encoded digital video signal 

an indication of the video display format for the decoded video signal; 

a low-pass filter, responsive to a first control signal, which operates on the 

frequency-domain transformed digital video signal to selectively cause the digital 

video signal decoder to produce a decoded digital video signal having a resolution 

less than the resolution of the extracted video display format; and 

signal processing circuitry, responsive to a second control signal, which 

decodes the filtered frequency domain encoded digital video signal to produce an 

output video signal; and 

a controller, coupled to receive data which identifies the predetermined video display format and the 
extracted video display format, the controller producing the control signals for the digital video signal 
decoder which cause the decoder to convert the encoded digital video signal corresponding to the 
extracted display format into the output video signal having the predetermined video display format. 

1 1 . A digital video signal conversion system according to claim 10, wherein: 

the digital video signal decoder further includes a programmable spatial filter which is responsive to a 
format conversion control signal to resample the decoded digital video signal provided by the signal 
processing circuitry to produce the output video signal having the predetermined video display format; and 
the controller produces the format conversion control signal. 

12. A digital video signal conversion system according to claim 11 , wherein the digital video signal is 
encoded using an encoding technique specified by the moving pictures experts group (MPEG) in which 
the encoded digital video signal is contained in a plurality of packetized elementary stream (PES) packets, 
each PES packet having a header, and the data retrieval circuitry of the digital video signal decoder 
includes means for extracting the video display format of the encoded digital video signal from the header 
of one of the PES packets. 

13. A digital video signal conversion system according to claim 11, wherein the digital video signal is 
encoded using an encoding technique specified by the moving pictures experts group (MPEG) in which 
the encoded digital video signal is contained in a digital bit-stream including sequence records, each 
sequence record having a header, and the data retrieval circuitry of the digital video signal decoder 
includes means for extracting the video display format of the encoded digital video signal from the header 
of one of the sequence record. 

14. A digital video signal conversion system according to claim 11 further including a user input device 
through which a user may enter the data which identifies the predetermined video display format. 

15. A digital video signal conversion system according to claim 13 wherein: 

the video signal decoder further includes an on-screen display processor which may be programmed to 
produce a plurality of output video signals, each corresponding to a respectively different one of the 
plurality of video display formats; and 

the controller includes means, responsive to a first user control signal provided via the user input device, 
to cause the on-screen display device to sequentially provide video signals having respectively different 
video display formats and responsive to a second selection signal provided via the user input device to 
define one of respective video display formats as the predetermined video display format. 

16. A digital video signal conversion system according to claim 1 1 further including a video display device 
having the predetermined video display format, wherein the video display device includes a register which 
holds an identification signal and the controller is coupled to the display device to read the identification 
signal and, use the identification signal to generate the data identifying the predetermined video display 
format. 
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17. A digital video signal conversion system for decoding and reformatting a baseband encoded digital 
vjdeo signal which represents a video display format having a first aspect ratio to produce a decoded 
video signal suitable for display in a predetermined video display format having a second aspect ratio, 
different from the first aspect ratio, the system comprising: 

a digital video signal decoder including: 

an input terminal coupled to receive the encoded digital video signal; 

data retrieval circuitry which extracts from the encoded digital video signal 

an indication of the aspect ratio for the decoded video signal; and 

signal processing circuitry which decodes the encoded digital video signal 

to produce an output video signal, wherein the decoder is responsive to a control 

signal to process the encoded digital video signal according to respective first and 

second down conversion processes to produce an output signal in the 

predetermined video display format having respective first and second conversions 

between the first aspect ratio and the second aspect ratio; and 

a controller, coupled to receive data which identifies the extracted video display format, the controller 
producing control signals for the digital video signal decoder which cause the decoder to sequentially 
convert the decoded video signal having the first aspect ratio into a first and second output signals having, 
respectively, the first and second conversions between the first aspect ratio and the second aspect ratio. 

18. A digital video signal conversion system according to claim 17, wherein the first aspect ratio is wider 
than the second aspect ratio and the first and second conversions use respective first and second 
compression techniques to produce the output signal having the second aspect ratio. 

19 A digital video signal conversion system according to claim 17, wherein the first aspect ratio is narrower 
than the second aspect ratio and the first and second conversions use respective first and second 
expansion techniques to produce the output signal having the second aspect ratio. 
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